Skip to contentSkip to conversation

ftrace, perf: Adding support to use function trace

RFC NAK Jiri Olsa 5 participants 185 messages Nov 27, 2011 – Feb 17, 2012 Page 1 of 2

Subsystems (3)
lore
Share & tools
lore

Jiri Olsa

author
#0 lore raw

hi,
here's another version of perf support for function trace with filter. The changeset is working and hopefully is not introducing more bugs.. ;) still testing.

Patch 1 through 3 could be probably taken separately as they fix independent issues. The rest is specific to the perf function tracing.

I needed a way to temporarily enable/disable ftrace_ops based on the event in/out scheduling, patch 4/9 is introducing that. Not sure this is the best way though..

Also adding open/close, add/del registration actions for tracepoint (patches 5,6) to have suitable place to control the function trace.

Patch 7 enables the function event perf registration, and patches 8 and 9 add filter support.

attached patches:
- 1/9 trace: Fix uninitialized variable compiler warning
- 2/9 ftrace: Fix possible NULL dereferencing in __ftrace_hash_rec_update
- 3/9 ftrace: Fix shutdown to disable calls properly
- 4/9 ftrace: Add enable/disable ftrace_ops control interface
- 5/9 ftrace, perf: Add open/close tracepoint perf registration actions
- 6/9 ftrace, perf: Add add/del tracepoint perf registration actions
- 7/9 ftrace, perf: Add support to use function tracepoint in perf
- 8/9 ftrace, perf: Add FILTER_TRACE_FN event field type
- 9/9 ftrace, perf: Add filter support for function trace event

thanks for comments,
jirka

---
 include/linux/ftrace.h             |   12 ++
 include/linux/ftrace_event.h       |    9 ++-
 include/linux/perf_event.h         |    3 +
 kernel/trace/ftrace.c              |   45 ++++++---
 kernel/trace/trace.c               |    3 +-
 kernel/trace/trace.h               |    9 ++-
 kernel/trace/trace_event_perf.c    |  208 ++++++++++++++++++++++++++++-------
 kernel/trace/trace_events.c        |   12 ++-
 kernel/trace/trace_events_filter.c |  114 ++++++++++++++++++--
 kernel/trace/trace_export.c        |   53 ++++++++-
 kernel/trace/trace_kprobe.c        |    8 ++-
 kernel/trace/trace_syscalls.c      |   18 +++-
 12 files changed, 413 insertions(+), 81 deletions(-)
author
#1 lore raw

PATCH 1/9 trace: Fix uninitialized variable compiler warning

Initialize page2 variable to make compiler happy.

---
 kernel/trace/trace.c |    3 +--
 1 files changed, 1 insertions(+), 2 deletions(-)
kernel/trace/trace.c +1 -2
index 9e158cc..4a06862 100644
@@ -3655,8 +3655,7 @@ tracing_mark_write(struct file *filp, const char __user *ubuf,
 	struct page *pages[2];
 	int nr_pages = 1;
 	ssize_t written;
-	void *page1;
-	void *page2;
+	void *page1, *page2 = NULL;
 	int offset;
 	int size;
 	int len;
Signature · 1 line
1.7.1
maintainer Nacked-by on v1 1/9 NAK
#2 lore raw

Re: [PATCH 1/9] trace: Fix uninitialized variable compiler warning

Quoted 1 line · Jiri Olsa

Initialize page2 variable to make compiler happy.

What compiler is this? Because this is a compiler bug. In fact, there's no check for page2 being NULL, so if it is used uninitialized it will crash the kernel. I don't like these "make the compiler shut up" fixes, because honestly, changes like this hide bugs.

-- Steve

Quoted 20 lines
---
 kernel/trace/trace.c |    3 +--
 1 files changed, 1 insertions(+), 2 deletions(-)
kernel/trace/trace.c +1 -2
index 9e158cc..4a06862 100644
@@ -3655,8 +3655,7 @@ tracing_mark_write(struct file *filp, const char __user *ubuf,
 	struct page *pages[2];
 	int nr_pages = 1;
 	ssize_t written;
-	void *page1;
-	void *page2;
+	void *page1, *page2 = NULL;
 	int offset;
 	int size;
 	int len;
author
#3 lore raw

Re: [PATCH 1/9] trace: Fix uninitialized variable compiler warning

Quoted 7 lines · Steven Rostedt
Quoted 1 line · Jiri Olsa

Initialize page2 variable to make compiler happy.

What compiler is this? Because this is a compiler bug. In fact, there's no check for page2 being NULL, so if it is used uninitialized it will crash the kernel. I don't like these "make the compiler shut up" fixes, because honestly, changes like this hide bugs.

[jolsa@krava1 ~]$ gcc --version
gcc (GCC) 4.4.5 20110214 (Red Hat 4.4.5-6)

Quoted 2 lines

understood, np

jirka

Quoted 25 lines

-- Steve

Quoted 20 lines
---
 kernel/trace/trace.c |    3 +--
 1 files changed, 1 insertions(+), 2 deletions(-)
kernel/trace/trace.c +1 -2
index 9e158cc..4a06862 100644
@@ -3655,8 +3655,7 @@ tracing_mark_write(struct file *filp, const char __user *ubuf,
 	struct page *pages[2];
 	int nr_pages = 1;
 	ssize_t written;
-	void *page1;
-	void *page2;
+	void *page1, *page2 = NULL;
 	int offset;
 	int size;
 	int len;
maintainer
#4 lore raw

Re: [PATCH 1/9] trace: Fix uninitialized variable compiler warning

Quoted 5 lines · Jiri Olsa
Quoted 3 lines · Steven Rostedt
Quoted 1 line · Jiri Olsa

Initialize page2 variable to make compiler happy.

Quoted 7 lines
Quoted 4 lines

What compiler is this? Because this is a compiler bug. In fact, there's no check for page2 being NULL, so if it is used uninitialized it will crash the kernel. I don't like these "make the compiler shut up" fixes, because honestly, changes like this hide bugs.

[jolsa@krava1 ~]$ gcc --version
gcc (GCC) 4.4.5 20110214 (Red Hat 4.4.5-6)

Yep, that's an old compiler. The newer ones don't show this as an error.

-- Steve

author
#5 lore raw

PATCH 2/9 ftrace: Fix possible NULL dereferencing in __ftrace_hash_rec_update

In reply to Jiri Olsa

We need to check the existence of the other_hash before we touch its count variable.

This issue is hit only when non global ftrace_ops is used. The global ftrace_ops is initialized with empty hashes.

---
 kernel/trace/ftrace.c |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)
kernel/trace/ftrace.c +2 -1
index b1e8943..c6d0293 100644
@@ -1372,7 +1372,8 @@ static void __ftrace_hash_rec_update(struct ftrace_ops *ops,
 			if (filter_hash && in_hash && !in_other_hash)
 				match = 1;
 			else if (!filter_hash && in_hash &&
-				 (in_other_hash || !other_hash->count))
+				 (in_other_hash ||
+				  !other_hash || !other_hash->count))
 				match = 1;
 		}
 		if (!match)
Signature · 1 line
1.7.1
maintainer
#6 lore raw

Re: [PATCH 2/9] ftrace: Fix possible NULL dereferencing in __ftrace_hash_rec_update

Quoted 16 lines · Jiri Olsa

We need to check the existence of the other_hash before we touch its count variable.

This issue is hit only when non global ftrace_ops is used. The global ftrace_ops is initialized with empty hashes.

---
 kernel/trace/ftrace.c |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)
kernel/trace/ftrace.c +0 -0
index b1e8943..c6d0293 100644

kernel/trace/ftrace.c static void __ftrace_hash_rec_update(struct ftrace_ops *ops, · line 1376

Quoted 6 lines
kernel/trace/ftrace.c +2 -1
@@ -1372,7 +1372,8 @@ static void __ftrace_hash_rec_update(struct ftrace_ops *ops,
 			if (filter_hash && in_hash && !in_other_hash)
 				match = 1;
 			else if (!filter_hash && in_hash &&
-				 (in_other_hash || !other_hash->count))
+				 (in_other_hash ||
+				  !other_hash || !other_hash->count))

Thanks! I hit this bug in too many places. I need to make a helper routine, which I think I will, that is:

hash_has_contents(hash)

that does the check for us.

-- Steve

Quoted 3 lines
kernel/trace/ftrace.c +0 -0
 				match = 1;
 		}
 		if (!match)
author
#7 lore raw

PATCH 3/9 ftrace: Fix shutdown to disable calls properly

In reply to Jiri Olsa

Each ftrace_startup call increases the call record's flag, so we need allways to decrease it when shutting down the ftrace_ops.

---
 kernel/trace/ftrace.c |    3 +--
 1 files changed, 1 insertions(+), 2 deletions(-)
kernel/trace/ftrace.c +1 -2
index c6d0293..0ca0c0d 100644
@@ -1744,8 +1744,7 @@ static void ftrace_shutdown(struct ftrace_ops *ops, int command)
 	if (ops != &global_ops || !global_start_up)
 		ops->flags &= ~FTRACE_OPS_FL_ENABLED;
 
-	if (!ftrace_start_up)
-		command |= FTRACE_DISABLE_CALLS;
+	command |= FTRACE_DISABLE_CALLS;
 
 	if (saved_ftrace_func != ftrace_trace_function) {
 		saved_ftrace_func = ftrace_trace_function;
Signature · 1 line
1.7.1
maintainer Nacked-by on v1 3/9 NAK
#8 lore raw

Re: [PATCH 3/9] ftrace: Fix shutdown to disable calls properly

Quoted 3 lines · Jiri Olsa

Each ftrace_startup call increases the call record's flag, so we need allways to decrease it when shutting down the ftrace_ops.

No, that's not how this works. I probably should comment this code better, because it caused me to reread it too ;)

Quoted 11 lines
---
 kernel/trace/ftrace.c |    3 +--
 1 files changed, 1 insertions(+), 2 deletions(-)
kernel/trace/ftrace.c +0 -0
index c6d0293..0ca0c0d 100644

kernel/trace/ftrace.c static void ftrace_shutdown(struct ftrace_ops *ops, int command) · line 1747

Quoted 6 lines
kernel/trace/ftrace.c +1 -2
@@ -1744,8 +1744,7 @@ static void ftrace_shutdown(struct ftrace_ops *ops, int command)
 	if (ops != &global_ops || !global_start_up)
 		ops->flags &= ~FTRACE_OPS_FL_ENABLED;
 
-	if (!ftrace_start_up)
-		command |= FTRACE_DISABLE_CALLS;
+	command |= FTRACE_DISABLE_CALLS;

FTRACE_DISABLE_CALLS will disable *all* functions for all tracers. If you are tracing with ftrace and perf, and one calls this with FTRACE_DISABLE_CALLS then both will no long be tracing anything.

When you call unregister_ftrace_function() it will disable the functions that you have enabled by the ops.

-- Steve

Quoted 3 lines
kernel/trace/ftrace.c +0 -0
 	if (saved_ftrace_func != ftrace_trace_function) {
 		saved_ftrace_func = ftrace_trace_function;
author
#9 lore raw

Re: [PATCH 3/9] ftrace: Fix shutdown to disable calls properly

Quoted 30 lines · Steven Rostedt
Quoted 3 lines · Jiri Olsa

Each ftrace_startup call increases the call record's flag, so we need allways to decrease it when shutting down the ftrace_ops.

No, that's not how this works. I probably should comment this code better, because it caused me to reread it too ;)

Quoted 11 lines
---
 kernel/trace/ftrace.c |    3 +--
 1 files changed, 1 insertions(+), 2 deletions(-)
kernel/trace/ftrace.c +0 -0
index c6d0293..0ca0c0d 100644

kernel/trace/ftrace.c static void ftrace_shutdown(struct ftrace_ops *ops, int command) · line 1747

Quoted 6 lines
kernel/trace/ftrace.c +1 -2
@@ -1744,8 +1744,7 @@ static void ftrace_shutdown(struct ftrace_ops *ops, int command)
 	if (ops != &global_ops || !global_start_up)
 		ops->flags &= ~FTRACE_OPS_FL_ENABLED;
 
-	if (!ftrace_start_up)
-		command |= FTRACE_DISABLE_CALLS;
+	command |= FTRACE_DISABLE_CALLS;

FTRACE_DISABLE_CALLS will disable *all* functions for all tracers. If you are tracing with ftrace and perf, and one calls this with FTRACE_DISABLE_CALLS then both will no long be tracing anything.

Quoted 4 lines

When you call unregister_ftrace_function() it will disable the functions that you have enabled by the ops.

ops, should read/test more carefully.. however the problem stays, here's one possible scenario of the issue:

- enable function trace like:
echo "mm_*" > ./set_ftrace_filter
echo function > ./current_tracer
- run:
./perf record -e ftrace:function cal
- this runs: register_ftrace_function/unregister_ftrace_function

- bad result: after perf is finished, function trace will have all the functions enabled

The reason is, that after the perf record calls unregister_ftrace_function, only the functions' record flags (struct dyn_ftrace::flags) are changed, but functions' mcount calls are not replaced accordingly.

In our case, when function trace becomes again the only tracer, it will be plugged directly into the entry_BIT.S code, and thus getting all the functions from previous perf ftrace_ops without any filtering.

I think we need to project the function record flags state into the function call replacement during the unregister_ftrace_function.

jirka

Quoted 9 lines

-- Steve

Quoted 3 lines
kernel/trace/ftrace.c +0 -0
 	if (saved_ftrace_func != ftrace_trace_function) {
 		saved_ftrace_func = ftrace_trace_function;
author
#10 lore raw

PATCH 4/9 ftrace: Add enable/disable ftrace_ops control interface

In reply to Jiri Olsa

Adding a way to temporarily enable/disable ftrace_ops.

When there is a ftrace_ops with FTRACE_OPS_FL_CONTROL flag registered, the ftrace_ops_list_func processing function is used as ftrace function in order to have all registered ftrace_ops under control.

Also using jump label not to introduce overhead to current ftrace_ops_list_func processing.

---
 include/linux/ftrace.h |   12 ++++++++++++
 kernel/trace/ftrace.c  |   39 +++++++++++++++++++++++++++++----------
 2 files changed, 41 insertions(+), 10 deletions(-)
include/linux/ftrace.h +12 -0
index 26eafce..28b59f1 100644
@@ -35,12 +35,14 @@ enum {
 	FTRACE_OPS_FL_ENABLED		= 1 << 0,
 	FTRACE_OPS_FL_GLOBAL		= 1 << 1,
 	FTRACE_OPS_FL_DYNAMIC		= 1 << 2,
+	FTRACE_OPS_FL_CONTROL		= 1 << 3,
 };
 
 struct ftrace_ops {
 	ftrace_func_t			func;
 	struct ftrace_ops		*next;
 	unsigned long			flags;
+	atomic_t			disabled;
 #ifdef CONFIG_DYNAMIC_FTRACE
 	struct ftrace_hash		*notrace_hash;
 	struct ftrace_hash		*filter_hash;
@@ -97,6 +99,16 @@ int register_ftrace_function(struct ftrace_ops *ops);
 int unregister_ftrace_function(struct ftrace_ops *ops);
 void clear_ftrace_function(void);
 
+static inline void enable_ftrace_function(struct ftrace_ops *ops)
+{
+	atomic_dec(&ops->disabled);
+}
+
+static inline void disable_ftrace_function(struct ftrace_ops *ops)
+{
+	atomic_inc(&ops->disabled);
+}
+
 extern void ftrace_stub(unsigned long a0, unsigned long a1);
 
 #else /* !CONFIG_FUNCTION_TRACER */
kernel/trace/ftrace.c +29 -10
index 0ca0c0d..e5a9498 100644
@@ -30,6 +30,7 @@
 #include <linux/list.h>
 #include <linux/hash.h>
 #include <linux/rcupdate.h>
+#include <linux/jump_label.h>
 
 #include <trace/events/sched.h>
 
@@ -94,6 +95,8 @@ ftrace_func_t __ftrace_trace_function __read_mostly = ftrace_stub;
 ftrace_func_t ftrace_pid_function __read_mostly = ftrace_stub;
 static struct ftrace_ops global_ops;
 
+static struct jump_label_key ftrace_ops_control;
+
 static void
 ftrace_ops_list_func(unsigned long ip, unsigned long parent_ip);
 
@@ -196,17 +199,21 @@ static void update_ftrace_function(void)
 
 	update_global_ops();
 
-	/*
-	 * If we are at the end of the list and this ops is
-	 * not dynamic, then have the mcount trampoline call
-	 * the function directly
-	 */
-	if (ftrace_ops_list == &ftrace_list_end ||
-	    (ftrace_ops_list->next == &ftrace_list_end &&
-	     !(ftrace_ops_list->flags & FTRACE_OPS_FL_DYNAMIC)))
-		func = ftrace_ops_list->func;
-	else
+	if (jump_label_enabled(&ftrace_ops_control))
 		func = ftrace_ops_list_func;
+	else {
+		/*
+		 * If we are at the end of the list and this ops is
+		 * not dynamic, then have the mcount trampoline call
+		 * the function directly
+		 */
+		if (ftrace_ops_list == &ftrace_list_end ||
+		    (ftrace_ops_list->next == &ftrace_list_end &&
+		     !(ftrace_ops_list->flags & FTRACE_OPS_FL_DYNAMIC)))
+			func = ftrace_ops_list->func;
+		else
+			func = ftrace_ops_list_func;
+	}
 
 #ifdef CONFIG_HAVE_FUNCTION_TRACE_MCOUNT_TEST
 	ftrace_trace_function = func;
@@ -280,6 +287,9 @@ static int __register_ftrace_function(struct ftrace_ops *ops)
 	} else
 		add_ftrace_ops(&ftrace_ops_list, ops);
 
+	if (ops->flags & FTRACE_OPS_FL_CONTROL)
+		jump_label_inc(&ftrace_ops_control);
+
 	if (ftrace_enabled)
 		update_ftrace_function();
 
@@ -311,6 +321,9 @@ static int __unregister_ftrace_function(struct ftrace_ops *ops)
 	if (ret < 0)
 		return ret;
 
+	if (ops->flags & FTRACE_OPS_FL_CONTROL)
+		jump_label_dec(&ftrace_ops_control);
+
 	if (ftrace_enabled)
 		update_ftrace_function();
 
@@ -3577,8 +3590,14 @@ ftrace_ops_list_func(unsigned long ip, unsigned long parent_ip)
 	preempt_disable_notrace();
 	op = rcu_dereference_raw(ftrace_ops_list);
 	while (op != &ftrace_list_end) {
+		if (static_branch(&ftrace_ops_control))
+			if ((op->flags & FTRACE_OPS_FL_CONTROL) &&
+			    atomic_read(&op->disabled))
+				goto next;
+
 		if (ftrace_ops_test(op, ip))
 			op->func(ip, parent_ip);
+ next:
 		op = rcu_dereference_raw(op->next);
 	};
 	preempt_enable_notrace();
Signature · 1 line
1.7.1
maintainer
#11 lore raw

Re: [PATCH 4/9] ftrace: Add enable/disable ftrace_ops control interface

[ Added Peter Z ]

Quoted 10 lines · Jiri Olsa

Adding a way to temporarily enable/disable ftrace_ops.

When there is a ftrace_ops with FTRACE_OPS_FL_CONTROL flag registered, the ftrace_ops_list_func processing function is used as ftrace function in order to have all registered ftrace_ops under control.

Also using jump label not to introduce overhead to current ftrace_ops_list_func processing.

Are jump labels safe in NMI context yet? If not, this will need to wait till we make it so.

-- Steve

Quoted 130 lines
---
 include/linux/ftrace.h |   12 ++++++++++++
 kernel/trace/ftrace.c  |   39 +++++++++++++++++++++++++++++----------
 2 files changed, 41 insertions(+), 10 deletions(-)
include/linux/ftrace.h +12 -0
index 26eafce..28b59f1 100644
@@ -35,12 +35,14 @@ enum {
 	FTRACE_OPS_FL_ENABLED		= 1 << 0,
 	FTRACE_OPS_FL_GLOBAL		= 1 << 1,
 	FTRACE_OPS_FL_DYNAMIC		= 1 << 2,
+	FTRACE_OPS_FL_CONTROL		= 1 << 3,
 };
 
 struct ftrace_ops {
 	ftrace_func_t			func;
 	struct ftrace_ops		*next;
 	unsigned long			flags;
+	atomic_t			disabled;
 #ifdef CONFIG_DYNAMIC_FTRACE
 	struct ftrace_hash		*notrace_hash;
 	struct ftrace_hash		*filter_hash;
@@ -97,6 +99,16 @@ int register_ftrace_function(struct ftrace_ops *ops);
 int unregister_ftrace_function(struct ftrace_ops *ops);
 void clear_ftrace_function(void);
 
+static inline void enable_ftrace_function(struct ftrace_ops *ops)
+{
+	atomic_dec(&ops->disabled);
+}
+
+static inline void disable_ftrace_function(struct ftrace_ops *ops)
+{
+	atomic_inc(&ops->disabled);
+}
+
 extern void ftrace_stub(unsigned long a0, unsigned long a1);
 
 #else /* !CONFIG_FUNCTION_TRACER */
kernel/trace/ftrace.c +29 -10
index 0ca0c0d..e5a9498 100644
@@ -30,6 +30,7 @@
 #include <linux/list.h>
 #include <linux/hash.h>
 #include <linux/rcupdate.h>
+#include <linux/jump_label.h>
 
 #include <trace/events/sched.h>
 
@@ -94,6 +95,8 @@ ftrace_func_t __ftrace_trace_function __read_mostly = ftrace_stub;
 ftrace_func_t ftrace_pid_function __read_mostly = ftrace_stub;
 static struct ftrace_ops global_ops;
 
+static struct jump_label_key ftrace_ops_control;
+
 static void
 ftrace_ops_list_func(unsigned long ip, unsigned long parent_ip);
 
@@ -196,17 +199,21 @@ static void update_ftrace_function(void)
 
 	update_global_ops();
 
-	/*
-	 * If we are at the end of the list and this ops is
-	 * not dynamic, then have the mcount trampoline call
-	 * the function directly
-	 */
-	if (ftrace_ops_list == &ftrace_list_end ||
-	    (ftrace_ops_list->next == &ftrace_list_end &&
-	     !(ftrace_ops_list->flags & FTRACE_OPS_FL_DYNAMIC)))
-		func = ftrace_ops_list->func;
-	else
+	if (jump_label_enabled(&ftrace_ops_control))
 		func = ftrace_ops_list_func;
+	else {
+		/*
+		 * If we are at the end of the list and this ops is
+		 * not dynamic, then have the mcount trampoline call
+		 * the function directly
+		 */
+		if (ftrace_ops_list == &ftrace_list_end ||
+		    (ftrace_ops_list->next == &ftrace_list_end &&
+		     !(ftrace_ops_list->flags & FTRACE_OPS_FL_DYNAMIC)))
+			func = ftrace_ops_list->func;
+		else
+			func = ftrace_ops_list_func;
+	}
 
 #ifdef CONFIG_HAVE_FUNCTION_TRACE_MCOUNT_TEST
 	ftrace_trace_function = func;
@@ -280,6 +287,9 @@ static int __register_ftrace_function(struct ftrace_ops *ops)
 	} else
 		add_ftrace_ops(&ftrace_ops_list, ops);
 
+	if (ops->flags & FTRACE_OPS_FL_CONTROL)
+		jump_label_inc(&ftrace_ops_control);
+
 	if (ftrace_enabled)
 		update_ftrace_function();
 
@@ -311,6 +321,9 @@ static int __unregister_ftrace_function(struct ftrace_ops *ops)
 	if (ret < 0)
 		return ret;
 
+	if (ops->flags & FTRACE_OPS_FL_CONTROL)
+		jump_label_dec(&ftrace_ops_control);
+
 	if (ftrace_enabled)
 		update_ftrace_function();
 
@@ -3577,8 +3590,14 @@ ftrace_ops_list_func(unsigned long ip, unsigned long parent_ip)
 	preempt_disable_notrace();
 	op = rcu_dereference_raw(ftrace_ops_list);
 	while (op != &ftrace_list_end) {
+		if (static_branch(&ftrace_ops_control))
+			if ((op->flags & FTRACE_OPS_FL_CONTROL) &&
+			    atomic_read(&op->disabled))
+				goto next;
+
 		if (ftrace_ops_test(op, ip))
 			op->func(ip, parent_ip);
+ next:
 		op = rcu_dereference_raw(op->next);
 	};
 	preempt_enable_notrace();
maintainer
#12 lore raw

Re: [PATCH 4/9] ftrace: Add enable/disable ftrace_ops control interface

Quoted 3 lines · Steven Rostedt

Are jump labels safe in NMI context yet? If not, this will need to wait till we make it so.

I don't think they are, we currently very much rely on that stop_machine() crap. NMIs can go straight through that.

I think you can make it work with the stop_machine()-less approach, because then the NMI will trap on the INT3 which will wait for completion, sync and resume the NMI.

That of course relies on the NMI vs IRET crap getting sorted.

But even then, that's highly arch specific and I'm not sure we can make all archs that support both jump_label and NMIs work.

maintainer
#13 lore raw

Re: [PATCH 4/9] ftrace: Add enable/disable ftrace_ops control interface

Quoted 11 lines · Peter Zijlstra
Quoted 3 lines · Steven Rostedt

Are jump labels safe in NMI context yet? If not, this will need to wait till we make it so.

I don't think they are, we currently very much rely on that stop_machine() crap. NMIs can go straight through that.

I think you can make it work with the stop_machine()-less approach, because then the NMI will trap on the INT3 which will wait for completion, sync and resume the NMI.

Note that this of course precludes ever using jump_labels from do_int3 handlers ;-)

maintainer
#14 lore raw

Re: [PATCH 4/9] ftrace: Add enable/disable ftrace_ops control interface

Quoted 2 lines · Peter Zijlstra

Note that this of course precludes ever using jump_labels from do_int3 handlers ;-)

Hmm, how many do_int3 handlers are there?

Oh, I bet we could trace kprobe handlers, and this would break that.

I'm a bit nervous about adding jump_labels into function tracing handlers.

-- Steve

maintainer
#15 lore raw

Re: [PATCH 4/9] ftrace: Add enable/disable ftrace_ops control interface

Quoted 2 lines · Steven Rostedt

Hmm, how many do_int3 handlers are there?

Dunno, but there's the regular SIGTRAP one I guess, there's kprobes and there's going to be uprobes. kgdb looks to be having one as well.

Quoted 2 lines

Oh, I bet we could trace kprobe handlers, and this would break that.

That would be lovely pain indeed ;-)

maintainer
#16 lore raw

Re: [PATCH 4/9] ftrace: Add enable/disable ftrace_ops control interface

In reply to Peter Zijlstra
Quoted 7 lines · Peter Zijlstra
Quoted 3 lines · Steven Rostedt

Are jump labels safe in NMI context yet? If not, this will need to wait till we make it so.

I don't think they are, we currently very much rely on that stop_machine() crap. NMIs can go straight through that.

Ftrace has a way around it, it would be trivial (well maybe ;) to add the same to jump labels.

Quoted 9 lines

I think you can make it work with the stop_machine()-less approach, because then the NMI will trap on the INT3 which will wait for completion, sync and resume the NMI.

That of course relies on the NMI vs IRET crap getting sorted.

But even then, that's highly arch specific and I'm not sure we can make all archs that support both jump_label and NMIs work.

Actually, from what I've been told, x86 seems to be the only arch that does crazy things with NMIs. Most the other archs do NMI when the system is dead. That is, there's no return to normal system processing once an NMI is hit.

Another thing is, most systems are OK in modifying code on SMP, and stop_machine isn't even needed for that. I have code to test this out on PowerPc, and it seems to work well.

-- Steve

maintainer
#17 lore raw

Re: [PATCH 4/9] ftrace: Add enable/disable ftrace_ops control interface

Quoted 4 lines · Steven Rostedt

Actually, from what I've been told, x86 seems to be the only arch that does crazy things with NMIs. Most the other archs do NMI when the system is dead. That is, there's no return to normal system processing once an NMI is hit.

Sparc64 implements effective NMIs by playing games with their interrupt priority levels. local_irq_disable() disable the lower 15 (0-14) levels only, and their PMU interrupts at level 15.

That generates an effective NMI (interrupt not blocked by local_irq_disable()).

maintainer
#18 lore raw

Re: [PATCH 4/9] ftrace: Add enable/disable ftrace_ops control interface

Quoted 6 lines · Peter Zijlstra
Quoted 4 lines · Steven Rostedt

Actually, from what I've been told, x86 seems to be the only arch that does crazy things with NMIs. Most the other archs do NMI when the system is dead. That is, there's no return to normal system processing once an NMI is hit.

Quoted 6 lines

Sparc64 implements effective NMIs by playing games with their interrupt priority levels. local_irq_disable() disable the lower 15 (0-14) levels only, and their PMU interrupts at level 15.

That generates an effective NMI (interrupt not blocked by local_irq_disable()).

[ Added David ]

I think it's ok on sparc to modify code on one processor while another processor is executing it. If not then we need the tricks that x86 does, but if its ok, then we don't even need stop_machine();

-- Steve

maintainer
#19 lore raw

Re: [PATCH 4/9] ftrace: Add enable/disable ftrace_ops control interface

In reply to Jiri Olsa
Quoted 13 lines
kernel/trace/ftrace.c +3 -0
@@ -311,6 +321,9 @@ static int __unregister_ftrace_function(struct ftrace_ops *ops)
 	if (ret < 0)
 		return ret;
 
+	if (ops->flags & FTRACE_OPS_FL_CONTROL)
+		jump_label_dec(&ftrace_ops_control);
+
 	if (ftrace_enabled)
 		update_ftrace_function();
 

kernel/trace/ftrace.c ftrace_ops_list_func(unsigned long ip, unsigned long parent_ip) · line 3593

Quoted 4 lines
kernel/trace/ftrace.c +1 -0
@@ -3577,8 +3590,14 @@ ftrace_ops_list_func(unsigned long ip, unsigned long parent_ip)
 	preempt_disable_notrace();
 	op = rcu_dereference_raw(ftrace_ops_list);
 	while (op != &ftrace_list_end) {
+		if (static_branch(&ftrace_ops_control))

Instead of doing a static_branch() here, which makes me really nervous, because this is called in function trace context, which has some strict rules of its own, and is probably prone to recursion, we could add another "ops" similar to the global_ops.

We could make a control_ops, and add all ops with the FTRACE_OPS_FL_CONTROL flag set to it. And then this function will have its own loop that it will check the disabled flag, for the ops registered to it.

This code doesn't need to be touched, we just add a layer of redirection for control ops and it will solve the jump_label issue.

-- Steve

Quoted 10 lines
kernel/trace/ftrace.c +5 -0
+			if ((op->flags & FTRACE_OPS_FL_CONTROL) &&
+			    atomic_read(&op->disabled))
+				goto next;
+
 		if (ftrace_ops_test(op, ip))
 			op->func(ip, parent_ip);
+ next:
 		op = rcu_dereference_raw(op->next);
 	};
 	preempt_enable_notrace();
author
#20 lore raw

Re: [PATCH 4/9] ftrace: Add enable/disable ftrace_ops control interface

Quoted 23 lines · Steven Rostedt
Quoted 13 lines
kernel/trace/ftrace.c +3 -0
@@ -311,6 +321,9 @@ static int __unregister_ftrace_function(struct ftrace_ops *ops)
 	if (ret < 0)
 		return ret;
 
+	if (ops->flags & FTRACE_OPS_FL_CONTROL)
+		jump_label_dec(&ftrace_ops_control);
+
 	if (ftrace_enabled)
 		update_ftrace_function();
 

kernel/trace/ftrace.c ftrace_ops_list_func(unsigned long ip, unsigned long parent_ip) · line 3593

Quoted 4 lines
kernel/trace/ftrace.c +1 -0
@@ -3577,8 +3590,14 @@ ftrace_ops_list_func(unsigned long ip, unsigned long parent_ip)
 	preempt_disable_notrace();
 	op = rcu_dereference_raw(ftrace_ops_list);
 	while (op != &ftrace_list_end) {
+		if (static_branch(&ftrace_ops_control))

Instead of doing a static_branch() here, which makes me really nervous, because this is called in function trace context, which has some strict rules of its own, and is probably prone to recursion, we could add another "ops" similar to the global_ops.

Quoted 7 lines

We could make a control_ops, and add all ops with the FTRACE_OPS_FL_CONTROL flag set to it. And then this function will have its own loop that it will check the disabled flag, for the ops registered to it.

This code doesn't need to be touched, we just add a layer of redirection for control ops and it will solve the jump_label issue.

sounds good, I'll make the change

thanks,
jirka

Quoted 15 lines

-- Steve

Quoted 10 lines
kernel/trace/ftrace.c +5 -0
+			if ((op->flags & FTRACE_OPS_FL_CONTROL) &&
+			    atomic_read(&op->disabled))
+				goto next;
+
 		if (ftrace_ops_test(op, ip))
 			op->func(ip, parent_ip);
+ next:
 		op = rcu_dereference_raw(op->next);
 	};
 	preempt_enable_notrace();
author
#21 lore raw

PATCH 5/9 ftrace, perf: Add open/close tracepoint perf registration actions

In reply to Jiri Olsa

Adding TRACE_REG_PERF_OPEN and TRACE_REG_PERF_CLOSE to differentiate register/unregister from open/close actions.

The register/unregister actions are invoked for the first/last tracepoint user when opening/closing the evet.

The open/close actions are invoked for each tracepoint user when opening/closing the event.

---
 include/linux/ftrace_event.h    |    6 +-
 kernel/trace/trace.h            |    5 ++
 kernel/trace/trace_event_perf.c |  116 +++++++++++++++++++++++++--------------
 kernel/trace/trace_events.c     |   10 ++-
 kernel/trace/trace_kprobe.c     |    6 ++-
 kernel/trace/trace_syscalls.c   |   14 +++-
 6 files changed, 106 insertions(+), 51 deletions(-)
include/linux/ftrace_event.h +4 -2
index c3da42d..195e360 100644
@@ -146,6 +146,8 @@ enum trace_reg {
 	TRACE_REG_UNREGISTER,
 	TRACE_REG_PERF_REGISTER,
 	TRACE_REG_PERF_UNREGISTER,
+	TRACE_REG_PERF_OPEN,
+	TRACE_REG_PERF_CLOSE,
 };
 
 struct ftrace_event_call;
@@ -157,7 +159,7 @@ struct ftrace_event_class {
 	void			*perf_probe;
 #endif
 	int			(*reg)(struct ftrace_event_call *event,
-				       enum trace_reg type);
+				       enum trace_reg type, void *data);
 	int			(*define_fields)(struct ftrace_event_call *);
 	struct list_head	*(*get_fields)(struct ftrace_event_call *);
 	struct list_head	fields;
@@ -165,7 +167,7 @@ struct ftrace_event_class {
 };
 
 extern int ftrace_event_reg(struct ftrace_event_call *event,
-			    enum trace_reg type);
+			    enum trace_reg type, void *data);
 
 enum {
 	TRACE_EVENT_FL_ENABLED_BIT,
kernel/trace/trace.h +5 -0
index f8ec229..c4330dc 100644
@@ -825,4 +825,9 @@ extern const char *__stop___trace_bprintk_fmt[];
 	FTRACE_ENTRY(call, struct_name, id, PARAMS(tstruct), PARAMS(print))
 #include "trace_entries.h"
 
+#ifdef CONFIG_PERF_EVENTS
+int perf_ftrace_event_register(struct ftrace_event_call *call,
+			       enum trace_reg type, void *data);
+#endif
+
 #endif /* _LINUX_KERNEL_TRACE_H */
kernel/trace/trace_event_perf.c +75 -41

154 lines not on this page · raw message

kernel/trace/trace_events.c +7 -3
index c212a7f..5138fea 100644
@@ -147,7 +147,8 @@ int trace_event_raw_init(struct ftrace_event_call *call)
 }
 EXPORT_SYMBOL_GPL(trace_event_raw_init);
 
-int ftrace_event_reg(struct ftrace_event_call *call, enum trace_reg type)
+int ftrace_event_reg(struct ftrace_event_call *call,
+		     enum trace_reg type, void *data)
 {
 	switch (type) {
 	case TRACE_REG_REGISTER:
@@ -170,6 +171,9 @@ int ftrace_event_reg(struct ftrace_event_call *call, enum trace_reg type)
 					    call->class->perf_probe,
 					    call);
 		return 0;
+	case TRACE_REG_PERF_OPEN:
+	case TRACE_REG_PERF_CLOSE:
+		return 0;
 #endif
 	}
 	return 0;
@@ -209,7 +213,7 @@ static int ftrace_event_enable_disable(struct ftrace_event_call *call,
 				tracing_stop_cmdline_record();
 				call->flags &= ~TRACE_EVENT_FL_RECORDED_CMD;
 			}
-			call->class->reg(call, TRACE_REG_UNREGISTER);
+			call->class->reg(call, TRACE_REG_UNREGISTER, NULL);
 		}
 		break;
 	case 1:
@@ -218,7 +222,7 @@ static int ftrace_event_enable_disable(struct ftrace_event_call *call,
 				tracing_start_cmdline_record();
 				call->flags |= TRACE_EVENT_FL_RECORDED_CMD;
 			}
-			ret = call->class->reg(call, TRACE_REG_REGISTER);
+			ret = call->class->reg(call, TRACE_REG_REGISTER, NULL);
 			if (ret) {
 				tracing_stop_cmdline_record();
 				pr_info("event trace: Could not enable event "
kernel/trace/trace_kprobe.c +5 -1
index 00d527c..5667f89 100644
@@ -1892,7 +1892,8 @@ static __kprobes void kretprobe_perf_func(struct kretprobe_instance *ri,
 #endif	/* CONFIG_PERF_EVENTS */
 
 static __kprobes
-int kprobe_register(struct ftrace_event_call *event, enum trace_reg type)
+int kprobe_register(struct ftrace_event_call *event,
+		    enum trace_reg type, void *data)
 {
 	struct trace_probe *tp = (struct trace_probe *)event->data;
 
@@ -1909,6 +1910,9 @@ int kprobe_register(struct ftrace_event_call *event, enum trace_reg type)
 	case TRACE_REG_PERF_UNREGISTER:
 		disable_trace_probe(tp, TP_FLAG_PROFILE);
 		return 0;
+	case TRACE_REG_PERF_OPEN:
+	case TRACE_REG_PERF_CLOSE:
+		return 0;
 #endif
 	}
 	return 0;
kernel/trace/trace_syscalls.c +10 -4
index cb65454..6916b0d 100644
@@ -17,9 +17,9 @@ static DECLARE_BITMAP(enabled_enter_syscalls, NR_syscalls);
 static DECLARE_BITMAP(enabled_exit_syscalls, NR_syscalls);
 
 static int syscall_enter_register(struct ftrace_event_call *event,
-				 enum trace_reg type);
+				 enum trace_reg type, void *data);
 static int syscall_exit_register(struct ftrace_event_call *event,
-				 enum trace_reg type);
+				 enum trace_reg type, void *data);
 
 static int syscall_enter_define_fields(struct ftrace_event_call *call);
 static int syscall_exit_define_fields(struct ftrace_event_call *call);
@@ -649,7 +649,7 @@ void perf_sysexit_disable(struct ftrace_event_call *call)
 #endif /* CONFIG_PERF_EVENTS */
 
 static int syscall_enter_register(struct ftrace_event_call *event,
-				 enum trace_reg type)
+				 enum trace_reg type, void *data)
 {
 	switch (type) {
 	case TRACE_REG_REGISTER:
@@ -664,13 +664,16 @@ static int syscall_enter_register(struct ftrace_event_call *event,
 	case TRACE_REG_PERF_UNREGISTER:
 		perf_sysenter_disable(event);
 		return 0;
+	case TRACE_REG_PERF_OPEN:
+	case TRACE_REG_PERF_CLOSE:
+		return 0;
 #endif
 	}
 	return 0;
 }
 
 static int syscall_exit_register(struct ftrace_event_call *event,
-				 enum trace_reg type)
+				 enum trace_reg type, void *data)
 {
 	switch (type) {
 	case TRACE_REG_REGISTER:
@@ -685,6 +688,9 @@ static int syscall_exit_register(struct ftrace_event_call *event,
 	case TRACE_REG_PERF_UNREGISTER:
 		perf_sysexit_disable(event);
 		return 0;
+	case TRACE_REG_PERF_OPEN:
+	case TRACE_REG_PERF_CLOSE:
+		return 0;
 #endif
 	}
 	return 0;
Signature · 1 line
1.7.1
author
#22 lore raw

PATCH 6/9 ftrace, perf: Add add/del tracepoint perf registration actions

In reply to Jiri Olsa

Adding TRACE_REG_PERF_ADD and TRACE_REG_PERF_DEL to handle perf event schedule in/out actions.

The add action is invoked for when the perf event is scheduled in, while the del action is invoked when the event is scheduled out.

---
 include/linux/ftrace_event.h    |    2 ++
 kernel/trace/trace_event_perf.c |    4 +++-
 kernel/trace/trace_events.c     |    2 ++
 kernel/trace/trace_kprobe.c     |    2 ++
 kernel/trace/trace_syscalls.c   |    4 ++++
 5 files changed, 13 insertions(+), 1 deletions(-)
include/linux/ftrace_event.h +2 -0
index 195e360..2bf677c 100644
@@ -148,6 +148,8 @@ enum trace_reg {
 	TRACE_REG_PERF_UNREGISTER,
 	TRACE_REG_PERF_OPEN,
 	TRACE_REG_PERF_CLOSE,
+	TRACE_REG_PERF_ADD,
+	TRACE_REG_PERF_DEL,
 };
 
 struct ftrace_event_call;
kernel/trace/trace_event_perf.c +3 -1
index 0cfcc37..d72af0b 100644
@@ -211,12 +211,14 @@ int perf_trace_add(struct perf_event *p_event, int flags)
 	list = this_cpu_ptr(pcpu_list);
 	hlist_add_head_rcu(&p_event->hlist_entry, list);
 
-	return 0;
+	return tp_event->class->reg(tp_event, TRACE_REG_PERF_ADD, p_event);
 }
 
 void perf_trace_del(struct perf_event *p_event, int flags)
 {
+	struct ftrace_event_call *tp_event = p_event->tp_event;
 	hlist_del_rcu(&p_event->hlist_entry);
+	tp_event->class->reg(tp_event, TRACE_REG_PERF_DEL, p_event);
 }
 
 __kprobes void *perf_trace_buf_prepare(int size, unsigned short type,
kernel/trace/trace_events.c +2 -0
index 5138fea..079a93a 100644
@@ -173,6 +173,8 @@ int ftrace_event_reg(struct ftrace_event_call *call,
 		return 0;
 	case TRACE_REG_PERF_OPEN:
 	case TRACE_REG_PERF_CLOSE:
+	case TRACE_REG_PERF_ADD:
+	case TRACE_REG_PERF_DEL:
 		return 0;
 #endif
 	}
kernel/trace/trace_kprobe.c +2 -0
index 5667f89..580a05e 100644
@@ -1912,6 +1912,8 @@ int kprobe_register(struct ftrace_event_call *event,
 		return 0;
 	case TRACE_REG_PERF_OPEN:
 	case TRACE_REG_PERF_CLOSE:
+	case TRACE_REG_PERF_ADD:
+	case TRACE_REG_PERF_DEL:
 		return 0;
 #endif
 	}
kernel/trace/trace_syscalls.c +4 -0
index 6916b0d..dbdd804 100644
@@ -666,6 +666,8 @@ static int syscall_enter_register(struct ftrace_event_call *event,
 		return 0;
 	case TRACE_REG_PERF_OPEN:
 	case TRACE_REG_PERF_CLOSE:
+	case TRACE_REG_PERF_ADD:
+	case TRACE_REG_PERF_DEL:
 		return 0;
 #endif
 	}
@@ -690,6 +692,8 @@ static int syscall_exit_register(struct ftrace_event_call *event,
 		return 0;
 	case TRACE_REG_PERF_OPEN:
 	case TRACE_REG_PERF_CLOSE:
+	case TRACE_REG_PERF_ADD:
+	case TRACE_REG_PERF_DEL:
 		return 0;
 #endif
 	}
Signature · 1 line
1.7.1
author
#23 lore raw

PATCH 7/9 ftrace, perf: Add support to use function tracepoint in perf

In reply to Jiri Olsa

Adding perf registration support for the ftrace function event, so it is now possible to register it via perf interface.

The perf_event struct statically contains ftrace_ops as a handle for function tracer. The function tracer is registered/unregistered in open/close actions, and enabled/disabled in add/del actions.

It is now possible to use function trace within perf commands like:

perf record -e ftrace:function ls
perf stat -e ftrace:function ls

---
 include/linux/perf_event.h      |    3 +
 kernel/trace/trace_event_perf.c |   88 +++++++++++++++++++++++++++++++++++++++
 kernel/trace/trace_export.c     |   23 ++++++++++
 3 files changed, 114 insertions(+), 0 deletions(-)
include/linux/perf_event.h +3 -0
index 1e9ebe5..6071995 100644
@@ -847,6 +847,9 @@ struct perf_event {
 #ifdef CONFIG_EVENT_TRACING
 	struct ftrace_event_call	*tp_event;
 	struct event_filter		*filter;
+#ifdef CONFIG_FUNCTION_TRACER
+	struct ftrace_ops               ftrace_ops;
+#endif
 #endif
 
 #ifdef CONFIG_CGROUP_PERF
kernel/trace/trace_event_perf.c +88 -0

91 lines not on this page · raw message

kernel/trace/trace_export.c +23 -0
index bbeec31..62e86a5 100644
@@ -131,6 +131,28 @@ ftrace_define_fields_##name(struct ftrace_event_call *event_call)	\
 
 #include "trace_entries.h"
 
+static int ftrace_event_class_register(struct ftrace_event_call *call,
+				       enum trace_reg type, void *data)
+{
+	switch (type) {
+	case TRACE_REG_PERF_REGISTER:
+	case TRACE_REG_PERF_UNREGISTER:
+		return 0;
+	case TRACE_REG_PERF_OPEN:
+	case TRACE_REG_PERF_CLOSE:
+	case TRACE_REG_PERF_ADD:
+	case TRACE_REG_PERF_DEL:
+#ifdef CONFIG_PERF_EVENTS
+		return perf_ftrace_event_register(call, type, data);
+#endif
+	case TRACE_REG_REGISTER:
+	case TRACE_REG_UNREGISTER:
+		break;
+	}
+
+	return -EINVAL;
+}
+
 #undef __entry
 #define __entry REC
 
@@ -159,6 +181,7 @@ struct ftrace_event_class event_class_ftrace_##call = {			\
 	.system			= __stringify(TRACE_SYSTEM),		\
 	.define_fields		= ftrace_define_fields_##call,		\
 	.fields			= LIST_HEAD_INIT(event_class_ftrace_##call.fields),\
+	.reg			= ftrace_event_class_register,		\
 };									\
 									\
 struct ftrace_event_call __used event_##call = {			\
Signature · 1 line
1.7.1
maintainer
#24 lore raw

Re: [PATCH 7/9] ftrace, perf: Add support to use function tracepoint in perf

Quoted 7 lines · Jiri Olsa

Adding perf registration support for the ftrace function event, so it is now possible to register it via perf interface.

The perf_event struct statically contains ftrace_ops as a handle for function tracer. The function tracer is registered/unregistered in open/close actions, and enabled/disabled in add/del actions.

Quoted 5 lines

It is now possible to use function trace within perf commands like:

perf record -e ftrace:function ls
perf stat -e ftrace:function ls

Question. This is a root only command, correct? Otherwise, we are allowing any user to create a large performance impact to the system.

Quoted 59 lines
---
 include/linux/perf_event.h      |    3 +
 kernel/trace/trace_event_perf.c |   88 +++++++++++++++++++++++++++++++++++++++
 kernel/trace/trace_export.c     |   23 ++++++++++
 3 files changed, 114 insertions(+), 0 deletions(-)
include/linux/perf_event.h +3 -0
index 1e9ebe5..6071995 100644
@@ -847,6 +847,9 @@ struct perf_event {
 #ifdef CONFIG_EVENT_TRACING
 	struct ftrace_event_call	*tp_event;
 	struct event_filter		*filter;
+#ifdef CONFIG_FUNCTION_TRACER
+	struct ftrace_ops               ftrace_ops;
+#endif
 #endif
 
 #ifdef CONFIG_CGROUP_PERF
kernel/trace/trace_event_perf.c +29 -0
index d72af0b..4be0f73 100644
@@ -250,3 +250,91 @@ __kprobes void *perf_trace_buf_prepare(int size, unsigned short type,
 	return raw_data;
 }
 EXPORT_SYMBOL_GPL(perf_trace_buf_prepare);
+
+
+static void
+perf_ftrace_function_call(unsigned long ip, unsigned long parent_ip)
+{
+	struct ftrace_entry *entry;
+	struct hlist_head *head;
+	struct pt_regs regs;
+	int rctx;
+
+#define ENTRY_SIZE (ALIGN(sizeof(struct ftrace_entry) + sizeof(u32), \
+		    sizeof(u64)) - sizeof(u32))
+
+	BUILD_BUG_ON(ENTRY_SIZE > PERF_MAX_TRACE_SIZE);
+
+	perf_fetch_caller_regs(&regs);
+
+	entry = perf_trace_buf_prepare(ENTRY_SIZE, TRACE_FN, NULL, &rctx);
+	if (!entry)
+		return;
+
+	entry->ip = ip;
+	entry->parent_ip = parent_ip;
+
+	head = this_cpu_ptr(event_function.perf_events);
+	perf_trace_buf_submit(entry, ENTRY_SIZE, rctx, 0,
+			      1, &regs, head);
+
+#undef ENTRY_SIZE

kernel/trace/trace_event_perf.c __kprobes void *perf_trace_buf_prepare(int size, unsigned short type, · line 291

Quoted 10 lines
kernel/trace/trace_event_perf.c +10 -0
+}
+
+static int perf_ftrace_function_register(struct perf_event *event)
+{
+	struct ftrace_ops *ops = &event->ftrace_ops;
+
+	ops->flags |= FTRACE_OPS_FL_CONTROL;
+	atomic_set(&ops->disabled, 1);
+	ops->func = perf_ftrace_function_call;
+	return register_ftrace_function(ops);

When is ADD called? Because as soon as you register this function, even though you have it "disabled" the system takes about a 13% impact on performance just by calling this.

kernel/trace/trace_event_perf.c __kprobes void *perf_trace_buf_prepare(int size, unsigned short type, · line 303

Quoted 12 lines
kernel/trace/trace_event_perf.c +12 -0
+}
+
+static int perf_ftrace_function_unregister(struct perf_event *event)
+{
+	struct ftrace_ops *ops = &event->ftrace_ops;
+	return unregister_ftrace_function(ops);
+}
+
+static void perf_ftrace_function_enable(struct perf_event *event)
+{
+	struct ftrace_ops *ops = &event->ftrace_ops;
+	enable_ftrace_function(ops);

Is it really an issue that we shouldn't call the full blown register instead? I'm not really understanding why this is a problem. Note, one of the improvements to ftrace in the near future is to enable ftrace without stop_machine.

-- Steve

Quoted 78 lines
kernel/trace/trace_event_perf.c +37 -0
+}
+
+static void perf_ftrace_function_disable(struct perf_event *event)
+{
+	struct ftrace_ops *ops = &event->ftrace_ops;
+	disable_ftrace_function(ops);
+}
+
+int perf_ftrace_event_register(struct ftrace_event_call *call,
+			       enum trace_reg type, void *data)
+{
+	int etype = call->event.type;
+
+	if (etype != TRACE_FN)
+		return -EINVAL;
+
+	switch (type) {
+	case TRACE_REG_REGISTER:
+	case TRACE_REG_UNREGISTER:
+		break;
+	case TRACE_REG_PERF_REGISTER:
+	case TRACE_REG_PERF_UNREGISTER:
+		return 0;
+	case TRACE_REG_PERF_OPEN:
+		return perf_ftrace_function_register(data);
+	case TRACE_REG_PERF_CLOSE:
+		return perf_ftrace_function_unregister(data);
+	case TRACE_REG_PERF_ADD:
+		perf_ftrace_function_enable(data);
+		return 0;
+	case TRACE_REG_PERF_DEL:
+		perf_ftrace_function_disable(data);
+		return 0;
+	}
+
+	return -EINVAL;
+}
kernel/trace/trace_export.c +23 -0
index bbeec31..62e86a5 100644
@@ -131,6 +131,28 @@ ftrace_define_fields_##name(struct ftrace_event_call *event_call)	\
 
 #include "trace_entries.h"
 
+static int ftrace_event_class_register(struct ftrace_event_call *call,
+				       enum trace_reg type, void *data)
+{
+	switch (type) {
+	case TRACE_REG_PERF_REGISTER:
+	case TRACE_REG_PERF_UNREGISTER:
+		return 0;
+	case TRACE_REG_PERF_OPEN:
+	case TRACE_REG_PERF_CLOSE:
+	case TRACE_REG_PERF_ADD:
+	case TRACE_REG_PERF_DEL:
+#ifdef CONFIG_PERF_EVENTS
+		return perf_ftrace_event_register(call, type, data);
+#endif
+	case TRACE_REG_REGISTER:
+	case TRACE_REG_UNREGISTER:
+		break;
+	}
+
+	return -EINVAL;
+}
+
 #undef __entry
 #define __entry REC
 
@@ -159,6 +181,7 @@ struct ftrace_event_class event_class_ftrace_##call = {			\
 	.system			= __stringify(TRACE_SYSTEM),		\
 	.define_fields		= ftrace_define_fields_##call,		\
 	.fields			= LIST_HEAD_INIT(event_class_ftrace_##call.fields),\
+	.reg			= ftrace_event_class_register,		\
 };									\
 									\
 struct ftrace_event_call __used event_##call = {			\
maintainer
#25 lore raw

Re: [PATCH 7/9] ftrace, perf: Add support to use function tracepoint in perf

Quoted 4 lines · Steven Rostedt
Quoted 1 line

perf stat -e ftrace:function ls

Question. This is a root only command, correct? Otherwise, we are allowing any user to create a large performance impact to the system.

Typically not, although I haven't looked at Jiri's implementation of the function tracepoint.

maintainer
#26 lore raw

Re: [PATCH 7/9] ftrace, perf: Add support to use function tracepoint in perf

Quoted 9 lines · Peter Zijlstra
Quoted 4 lines · Steven Rostedt
Quoted 1 line

perf stat -e ftrace:function ls

Question. This is a root only command, correct? Otherwise, we are allowing any user to create a large performance impact to the system.

Typically not, although I haven't looked at Jiri's implementation of the function tracepoint.

I would not allow function tracing for non root users. It's just way too invasive to let such a thing happen.

-- Steve

author
#27 lore raw

Re: [PATCH 7/9] ftrace, perf: Add support to use function tracepoint in perf

Quoted 7 lines · Steven Rostedt
Quoted 6 lines · Peter Zijlstra
Quoted 4 lines · Steven Rostedt
Quoted 1 line

perf stat -e ftrace:function ls

Question. This is a root only command, correct? Otherwise, we are allowing any user to create a large performance impact to the system.

Quoted 6 lines
Quoted 3 lines

Typically not, although I haven't looked at Jiri's implementation of the function tracepoint.

I would not allow function tracing for non root users. It's just way too invasive to let such a thing happen.

I believe it's for root only, I'll double check

jirka

Quoted 5 lines

-- Steve

maintainer
#28 lore raw

Re: [PATCH 7/9] ftrace, perf: Add support to use function tracepoint in perf

In reply to Steven Rostedt
Quoted 4 lines · Steven Rostedt
Quoted 4 lines
kernel/trace/trace_event_perf.c +4 -0
+static int perf_ftrace_function_register(struct perf_event *event)
+{
+     struct ftrace_ops *ops = &event->ftrace_ops;
+
Quoted 8 lines

kernel/trace/trace_event_perf.c __kprobes void *perf_trace_buf_prepare(int size, unsigned short type, · line 291

Quoted 4 lines
kernel/trace/trace_event_perf.c +4 -0
+     ops->flags |= FTRACE_OPS_FL_CONTROL;
+     atomic_set(&ops->disabled, 1);
+     ops->func = perf_ftrace_function_call;
+     return register_ftrace_function(ops);

When is ADD called? Because as soon as you register this function, even though you have it "disabled" the system takes about a 13% impact on performance just by calling this.

Typically at context switch time.

Quoted 9 lines

kernel/trace/trace_event_perf.c __kprobes void *perf_trace_buf_prepare(int size, unsigned short type, · line 303

Quoted 4 lines
kernel/trace/trace_event_perf.c +4 -0
+static void perf_ftrace_function_enable(struct perf_event *event)
+{
+     struct ftrace_ops *ops = &event->ftrace_ops;
+     enable_ftrace_function(ops);

Is it really an issue that we shouldn't call the full blown register instead? I'm not really understanding why this is a problem. Note, one of the improvements to ftrace in the near future is to enable ftrace without stop_machine.

Yeah, but still not something you want to do during context switches, right?

maintainer
#29 lore raw

Re: [PATCH 7/9] ftrace, perf: Add support to use function tracepoint in perf

Quoted 7 lines · Peter Zijlstra
Quoted 4 lines

Is it really an issue that we shouldn't call the full blown register instead? I'm not really understanding why this is a problem. Note, one of the improvements to ftrace in the near future is to enable ftrace without stop_machine.

Yeah, but still not something you want to do during context switches, right?

Also, that's x86, there's still archs that need stop_machine(), right?

maintainer
#30 lore raw

Re: [PATCH 7/9] ftrace, perf: Add support to use function tracepoint in perf

Quoted 4 lines · Peter Zijlstra
Quoted 2 lines

Yeah, but still not something you want to do during context switches, right?

Also, that's x86, there's still archs that need stop_machine(), right?

s390 may need it, but till we add a work around there too.

But yeah, if this happens at context switch, then we want the fast enable/disable that Jiri made.

-- Steve

maintainer
#31 lore raw

Re: [PATCH 7/9] ftrace, perf: Add support to use function tracepoint in perf

Quoted 11 lines · Steven Rostedt
Quoted 4 lines · Peter Zijlstra
Quoted 2 lines

Yeah, but still not something you want to do during context switches, right?

Also, that's x86, there's still archs that need stop_machine(), right?

s390 may need it, but till we add a work around there too.

But yeah, if this happens at context switch, then we want the fast enable/disable that Jiri made.

Right, so the way tracepoint->perf works currently is we enable the tracepoint when we create the event. The tracepoint handler runs a per-cpu hlist. perf then registers with the per-cpu hlist on context switch.

author
#32 lore raw

PATCH 8/9 ftrace, perf: Add FILTER_TRACE_FN event field type

In reply to Jiri Olsa

Adding FILTER_TRACE_FN event field type for function tracepoint event, so it can be properly recognized within filtering code.

---
 include/linux/ftrace_event.h       |    1 +
 kernel/trace/trace_events_filter.c |    7 ++++++-
 kernel/trace/trace_export.c        |   25 ++++++++++++++++++++-----
 3 files changed, 27 insertions(+), 6 deletions(-)
include/linux/ftrace_event.h +1 -0
index 2bf677c..dd478fc 100644
@@ -245,6 +245,7 @@ enum {
 	FILTER_STATIC_STRING,
 	FILTER_DYN_STRING,
 	FILTER_PTR_STRING,
+	FILTER_TRACE_FN,
 };
 
 #define EVENT_STORAGE_SIZE 128
kernel/trace/trace_events_filter.c +6 -1
index fdc6d22..7b0b04c 100644
@@ -900,6 +900,11 @@ int filter_assign_type(const char *type)
 	return FILTER_OTHER;
 }
 
+static bool is_function_field(struct ftrace_event_field *field)
+{
+	return field->filter_type == FILTER_TRACE_FN;
+}
+
 static bool is_string_field(struct ftrace_event_field *field)
 {
 	return field->filter_type == FILTER_DYN_STRING ||
@@ -987,7 +992,7 @@ static int init_pred(struct filter_parse_state *ps,
 			fn = filter_pred_strloc;
 		else
 			fn = filter_pred_pchar;
-	} else {
+	} else if (!is_function_field(field)) {
 		if (field->is_signed)
 			ret = strict_strtoll(pred->regex.pattern, 0, &val);
 		else
kernel/trace/trace_export.c +20 -5
index 62e86a5..7b035ab 100644
@@ -67,7 +67,7 @@ static void __always_unused ____ftrace_check_##name(void)	\
 	ret = trace_define_field(event_call, #type, #item,		\
 				 offsetof(typeof(field), item),		\
 				 sizeof(field.item),			\
-				 is_signed_type(type), FILTER_OTHER);	\
+				 is_signed_type(type), filter_type);	\
 	if (ret)							\
 		return ret;
 
@@ -77,7 +77,7 @@ static void __always_unused ____ftrace_check_##name(void)	\
 				 offsetof(typeof(field),		\
 					  container.item),		\
 				 sizeof(field.container.item),		\
-				 is_signed_type(type), FILTER_OTHER);	\
+				 is_signed_type(type), filter_type);	\
 	if (ret)							\
 		return ret;
 
@@ -91,7 +91,7 @@ static void __always_unused ____ftrace_check_##name(void)	\
 		ret = trace_define_field(event_call, event_storage, #item, \
 				 offsetof(typeof(field), item),		\
 				 sizeof(field.item),			\
-				 is_signed_type(type), FILTER_OTHER);	\
+				 is_signed_type(type), filter_type);	\
 		mutex_unlock(&event_storage_mutex);			\
 		if (ret)						\
 			return ret;					\
@@ -104,7 +104,7 @@ static void __always_unused ____ftrace_check_##name(void)	\
 				 offsetof(typeof(field),		\
 					  container.item),		\
 				 sizeof(field.container.item),		\
-				 is_signed_type(type), FILTER_OTHER);	\
+				 is_signed_type(type), filter_type);	\
 	if (ret)							\
 		return ret;
 
@@ -112,10 +112,24 @@ static void __always_unused ____ftrace_check_##name(void)	\
 #define __dynamic_array(type, item)					\
 	ret = trace_define_field(event_call, #type, #item,		\
 				 offsetof(typeof(field), item),		\
-				 0, is_signed_type(type), FILTER_OTHER);\
+				 0, is_signed_type(type), filter_type);\
 	if (ret)							\
 		return ret;
 
+#define FILTER_TYPE_TRACE_FN           FILTER_TRACE_FN
+#define FILTER_TYPE_TRACE_GRAPH_ENT    FILTER_OTHER
+#define FILTER_TYPE_TRACE_GRAPH_RET    FILTER_OTHER
+#define FILTER_TYPE_TRACE_CTX          FILTER_OTHER
+#define FILTER_TYPE_TRACE_WAKE         FILTER_OTHER
+#define FILTER_TYPE_TRACE_STACK                FILTER_OTHER
+#define FILTER_TYPE_TRACE_USER_STACK   FILTER_OTHER
+#define FILTER_TYPE_TRACE_BPRINT       FILTER_OTHER
+#define FILTER_TYPE_TRACE_PRINT                FILTER_OTHER
+#define FILTER_TYPE_TRACE_MMIO_RW      FILTER_OTHER
+#define FILTER_TYPE_TRACE_MMIO_MAP     FILTER_OTHER
+#define FILTER_TYPE_TRACE_BRANCH       FILTER_OTHER
+#define FILTER_TYPE(arg)               FILTER_TYPE_##arg
+
 #undef FTRACE_ENTRY
 #define FTRACE_ENTRY(name, struct_name, id, tstruct, print)		\
 int									\
@@ -123,6 +137,7 @@ ftrace_define_fields_##name(struct ftrace_event_call *event_call)	\
 {									\
 	struct struct_name field;					\
 	int ret;							\
+	int filter_type = FILTER_TYPE(id);				\
 									\
 	tstruct;							\
 									\
Signature · 1 line
1.7.1
maintainer
#33 lore raw

Re: [PATCH 8/9] ftrace, perf: Add FILTER_TRACE_FN event field type

BTW, Please Cc Peter Zijlstra too, as he maintains perf inside the kernel.

Quoted 2 lines · Jiri Olsa

Adding FILTER_TRACE_FN event field type for function tracepoint event, so it can be properly recognized within filtering code.

-ECHANGELOGTOOSHORT

I'm not sure what this is for.

-- Steve

Quoted 119 lines
---
 include/linux/ftrace_event.h       |    1 +
 kernel/trace/trace_events_filter.c |    7 ++++++-
 kernel/trace/trace_export.c        |   25 ++++++++++++++++++++-----
 3 files changed, 27 insertions(+), 6 deletions(-)
include/linux/ftrace_event.h +1 -0
index 2bf677c..dd478fc 100644
@@ -245,6 +245,7 @@ enum {
 	FILTER_STATIC_STRING,
 	FILTER_DYN_STRING,
 	FILTER_PTR_STRING,
+	FILTER_TRACE_FN,
 };
 
 #define EVENT_STORAGE_SIZE 128
kernel/trace/trace_events_filter.c +6 -1
index fdc6d22..7b0b04c 100644
@@ -900,6 +900,11 @@ int filter_assign_type(const char *type)
 	return FILTER_OTHER;
 }
 
+static bool is_function_field(struct ftrace_event_field *field)
+{
+	return field->filter_type == FILTER_TRACE_FN;
+}
+
 static bool is_string_field(struct ftrace_event_field *field)
 {
 	return field->filter_type == FILTER_DYN_STRING ||
@@ -987,7 +992,7 @@ static int init_pred(struct filter_parse_state *ps,
 			fn = filter_pred_strloc;
 		else
 			fn = filter_pred_pchar;
-	} else {
+	} else if (!is_function_field(field)) {
 		if (field->is_signed)
 			ret = strict_strtoll(pred->regex.pattern, 0, &val);
 		else
kernel/trace/trace_export.c +20 -5
index 62e86a5..7b035ab 100644
@@ -67,7 +67,7 @@ static void __always_unused ____ftrace_check_##name(void)	\
 	ret = trace_define_field(event_call, #type, #item,		\
 				 offsetof(typeof(field), item),		\
 				 sizeof(field.item),			\
-				 is_signed_type(type), FILTER_OTHER);	\
+				 is_signed_type(type), filter_type);	\
 	if (ret)							\
 		return ret;
 
@@ -77,7 +77,7 @@ static void __always_unused ____ftrace_check_##name(void)	\
 				 offsetof(typeof(field),		\
 					  container.item),		\
 				 sizeof(field.container.item),		\
-				 is_signed_type(type), FILTER_OTHER);	\
+				 is_signed_type(type), filter_type);	\
 	if (ret)							\
 		return ret;
 
@@ -91,7 +91,7 @@ static void __always_unused ____ftrace_check_##name(void)	\
 		ret = trace_define_field(event_call, event_storage, #item, \
 				 offsetof(typeof(field), item),		\
 				 sizeof(field.item),			\
-				 is_signed_type(type), FILTER_OTHER);	\
+				 is_signed_type(type), filter_type);	\
 		mutex_unlock(&event_storage_mutex);			\
 		if (ret)						\
 			return ret;					\
@@ -104,7 +104,7 @@ static void __always_unused ____ftrace_check_##name(void)	\
 				 offsetof(typeof(field),		\
 					  container.item),		\
 				 sizeof(field.container.item),		\
-				 is_signed_type(type), FILTER_OTHER);	\
+				 is_signed_type(type), filter_type);	\
 	if (ret)							\
 		return ret;
 
@@ -112,10 +112,24 @@ static void __always_unused ____ftrace_check_##name(void)	\
 #define __dynamic_array(type, item)					\
 	ret = trace_define_field(event_call, #type, #item,		\
 				 offsetof(typeof(field), item),		\
-				 0, is_signed_type(type), FILTER_OTHER);\
+				 0, is_signed_type(type), filter_type);\
 	if (ret)							\
 		return ret;
 
+#define FILTER_TYPE_TRACE_FN           FILTER_TRACE_FN
+#define FILTER_TYPE_TRACE_GRAPH_ENT    FILTER_OTHER
+#define FILTER_TYPE_TRACE_GRAPH_RET    FILTER_OTHER
+#define FILTER_TYPE_TRACE_CTX          FILTER_OTHER
+#define FILTER_TYPE_TRACE_WAKE         FILTER_OTHER
+#define FILTER_TYPE_TRACE_STACK                FILTER_OTHER
+#define FILTER_TYPE_TRACE_USER_STACK   FILTER_OTHER
+#define FILTER_TYPE_TRACE_BPRINT       FILTER_OTHER
+#define FILTER_TYPE_TRACE_PRINT                FILTER_OTHER
+#define FILTER_TYPE_TRACE_MMIO_RW      FILTER_OTHER
+#define FILTER_TYPE_TRACE_MMIO_MAP     FILTER_OTHER
+#define FILTER_TYPE_TRACE_BRANCH       FILTER_OTHER
+#define FILTER_TYPE(arg)               FILTER_TYPE_##arg
+
 #undef FTRACE_ENTRY
 #define FTRACE_ENTRY(name, struct_name, id, tstruct, print)		\
 int									\
@@ -123,6 +137,7 @@ ftrace_define_fields_##name(struct ftrace_event_call *event_call)	\
 {									\
 	struct struct_name field;					\
 	int ret;							\
+	int filter_type = FILTER_TYPE(id);				\
 									\
 	tstruct;							\
 									\
author
#34 lore raw

Re: [PATCH 8/9] ftrace, perf: Add FILTER_TRACE_FN event field type

Quoted 10 lines · Steven Rostedt

BTW, Please Cc Peter Zijlstra too, as he maintains perf inside the kernel.

Quoted 2 lines · Jiri Olsa

Adding FILTER_TRACE_FN event field type for function tracepoint event, so it can be properly recognized within filtering code.

-ECHANGELOGTOOSHORT

I'm not sure what this is for.

need a way to distinguish fucntion trace field for the next patch, I'll make this more fit or do it some other way.. ;)

thanks,
jirka

Quoted 124 lines

-- Steve

Quoted 119 lines
---
 include/linux/ftrace_event.h       |    1 +
 kernel/trace/trace_events_filter.c |    7 ++++++-
 kernel/trace/trace_export.c        |   25 ++++++++++++++++++++-----
 3 files changed, 27 insertions(+), 6 deletions(-)
include/linux/ftrace_event.h +1 -0
index 2bf677c..dd478fc 100644
@@ -245,6 +245,7 @@ enum {
 	FILTER_STATIC_STRING,
 	FILTER_DYN_STRING,
 	FILTER_PTR_STRING,
+	FILTER_TRACE_FN,
 };
 
 #define EVENT_STORAGE_SIZE 128
kernel/trace/trace_events_filter.c +6 -1
index fdc6d22..7b0b04c 100644
@@ -900,6 +900,11 @@ int filter_assign_type(const char *type)
 	return FILTER_OTHER;
 }
 
+static bool is_function_field(struct ftrace_event_field *field)
+{
+	return field->filter_type == FILTER_TRACE_FN;
+}
+
 static bool is_string_field(struct ftrace_event_field *field)
 {
 	return field->filter_type == FILTER_DYN_STRING ||
@@ -987,7 +992,7 @@ static int init_pred(struct filter_parse_state *ps,
 			fn = filter_pred_strloc;
 		else
 			fn = filter_pred_pchar;
-	} else {
+	} else if (!is_function_field(field)) {
 		if (field->is_signed)
 			ret = strict_strtoll(pred->regex.pattern, 0, &val);
 		else
kernel/trace/trace_export.c +20 -5
index 62e86a5..7b035ab 100644
@@ -67,7 +67,7 @@ static void __always_unused ____ftrace_check_##name(void)	\
 	ret = trace_define_field(event_call, #type, #item,		\
 				 offsetof(typeof(field), item),		\
 				 sizeof(field.item),			\
-				 is_signed_type(type), FILTER_OTHER);	\
+				 is_signed_type(type), filter_type);	\
 	if (ret)							\
 		return ret;
 
@@ -77,7 +77,7 @@ static void __always_unused ____ftrace_check_##name(void)	\
 				 offsetof(typeof(field),		\
 					  container.item),		\
 				 sizeof(field.container.item),		\
-				 is_signed_type(type), FILTER_OTHER);	\
+				 is_signed_type(type), filter_type);	\
 	if (ret)							\
 		return ret;
 
@@ -91,7 +91,7 @@ static void __always_unused ____ftrace_check_##name(void)	\
 		ret = trace_define_field(event_call, event_storage, #item, \
 				 offsetof(typeof(field), item),		\
 				 sizeof(field.item),			\
-				 is_signed_type(type), FILTER_OTHER);	\
+				 is_signed_type(type), filter_type);	\
 		mutex_unlock(&event_storage_mutex);			\
 		if (ret)						\
 			return ret;					\
@@ -104,7 +104,7 @@ static void __always_unused ____ftrace_check_##name(void)	\
 				 offsetof(typeof(field),		\
 					  container.item),		\
 				 sizeof(field.container.item),		\
-				 is_signed_type(type), FILTER_OTHER);	\
+				 is_signed_type(type), filter_type);	\
 	if (ret)							\
 		return ret;
 
@@ -112,10 +112,24 @@ static void __always_unused ____ftrace_check_##name(void)	\
 #define __dynamic_array(type, item)					\
 	ret = trace_define_field(event_call, #type, #item,		\
 				 offsetof(typeof(field), item),		\
-				 0, is_signed_type(type), FILTER_OTHER);\
+				 0, is_signed_type(type), filter_type);\
 	if (ret)							\
 		return ret;
 
+#define FILTER_TYPE_TRACE_FN           FILTER_TRACE_FN
+#define FILTER_TYPE_TRACE_GRAPH_ENT    FILTER_OTHER
+#define FILTER_TYPE_TRACE_GRAPH_RET    FILTER_OTHER
+#define FILTER_TYPE_TRACE_CTX          FILTER_OTHER
+#define FILTER_TYPE_TRACE_WAKE         FILTER_OTHER
+#define FILTER_TYPE_TRACE_STACK                FILTER_OTHER
+#define FILTER_TYPE_TRACE_USER_STACK   FILTER_OTHER
+#define FILTER_TYPE_TRACE_BPRINT       FILTER_OTHER
+#define FILTER_TYPE_TRACE_PRINT                FILTER_OTHER
+#define FILTER_TYPE_TRACE_MMIO_RW      FILTER_OTHER
+#define FILTER_TYPE_TRACE_MMIO_MAP     FILTER_OTHER
+#define FILTER_TYPE_TRACE_BRANCH       FILTER_OTHER
+#define FILTER_TYPE(arg)               FILTER_TYPE_##arg
+
 #undef FTRACE_ENTRY
 #define FTRACE_ENTRY(name, struct_name, id, tstruct, print)		\
 int									\
@@ -123,6 +137,7 @@ ftrace_define_fields_##name(struct ftrace_event_call *event_call)	\
 {									\
 	struct struct_name field;					\
 	int ret;							\
+	int filter_type = FILTER_TYPE(id);				\
 									\
 	tstruct;							\
 									\
author
#35 lore raw

Re: [PATCH 8/9] ftrace, perf: Add FILTER_TRACE_FN event field type

In reply to Steven Rostedt
Quoted 2 lines · Steven Rostedt

BTW, Please Cc Peter Zijlstra too, as he maintains perf inside the kernel.

thanks! ommited by mistake..

Quoted 132 lines · Jiri Olsa
Quoted 2 lines

Adding FILTER_TRACE_FN event field type for function tracepoint event, so it can be properly recognized within filtering code.

-ECHANGELOGTOOSHORT

I'm not sure what this is for.

-- Steve

Quoted 119 lines
---
 include/linux/ftrace_event.h       |    1 +
 kernel/trace/trace_events_filter.c |    7 ++++++-
 kernel/trace/trace_export.c        |   25 ++++++++++++++++++++-----
 3 files changed, 27 insertions(+), 6 deletions(-)
include/linux/ftrace_event.h +1 -0
index 2bf677c..dd478fc 100644
@@ -245,6 +245,7 @@ enum {
 	FILTER_STATIC_STRING,
 	FILTER_DYN_STRING,
 	FILTER_PTR_STRING,
+	FILTER_TRACE_FN,
 };
 
 #define EVENT_STORAGE_SIZE 128
kernel/trace/trace_events_filter.c +6 -1
index fdc6d22..7b0b04c 100644
@@ -900,6 +900,11 @@ int filter_assign_type(const char *type)
 	return FILTER_OTHER;
 }
 
+static bool is_function_field(struct ftrace_event_field *field)
+{
+	return field->filter_type == FILTER_TRACE_FN;
+}
+
 static bool is_string_field(struct ftrace_event_field *field)
 {
 	return field->filter_type == FILTER_DYN_STRING ||
@@ -987,7 +992,7 @@ static int init_pred(struct filter_parse_state *ps,
 			fn = filter_pred_strloc;
 		else
 			fn = filter_pred_pchar;
-	} else {
+	} else if (!is_function_field(field)) {
 		if (field->is_signed)
 			ret = strict_strtoll(pred->regex.pattern, 0, &val);
 		else
kernel/trace/trace_export.c +20 -5
index 62e86a5..7b035ab 100644
@@ -67,7 +67,7 @@ static void __always_unused ____ftrace_check_##name(void)	\
 	ret = trace_define_field(event_call, #type, #item,		\
 				 offsetof(typeof(field), item),		\
 				 sizeof(field.item),			\
-				 is_signed_type(type), FILTER_OTHER);	\
+				 is_signed_type(type), filter_type);	\
 	if (ret)							\
 		return ret;
 
@@ -77,7 +77,7 @@ static void __always_unused ____ftrace_check_##name(void)	\
 				 offsetof(typeof(field),		\
 					  container.item),		\
 				 sizeof(field.container.item),		\
-				 is_signed_type(type), FILTER_OTHER);	\
+				 is_signed_type(type), filter_type);	\
 	if (ret)							\
 		return ret;
 
@@ -91,7 +91,7 @@ static void __always_unused ____ftrace_check_##name(void)	\
 		ret = trace_define_field(event_call, event_storage, #item, \
 				 offsetof(typeof(field), item),		\
 				 sizeof(field.item),			\
-				 is_signed_type(type), FILTER_OTHER);	\
+				 is_signed_type(type), filter_type);	\
 		mutex_unlock(&event_storage_mutex);			\
 		if (ret)						\
 			return ret;					\
@@ -104,7 +104,7 @@ static void __always_unused ____ftrace_check_##name(void)	\
 				 offsetof(typeof(field),		\
 					  container.item),		\
 				 sizeof(field.container.item),		\
-				 is_signed_type(type), FILTER_OTHER);	\
+				 is_signed_type(type), filter_type);	\
 	if (ret)							\
 		return ret;
 
@@ -112,10 +112,24 @@ static void __always_unused ____ftrace_check_##name(void)	\
 #define __dynamic_array(type, item)					\
 	ret = trace_define_field(event_call, #type, #item,		\
 				 offsetof(typeof(field), item),		\
-				 0, is_signed_type(type), FILTER_OTHER);\
+				 0, is_signed_type(type), filter_type);\
 	if (ret)							\
 		return ret;
 
+#define FILTER_TYPE_TRACE_FN           FILTER_TRACE_FN
+#define FILTER_TYPE_TRACE_GRAPH_ENT    FILTER_OTHER
+#define FILTER_TYPE_TRACE_GRAPH_RET    FILTER_OTHER
+#define FILTER_TYPE_TRACE_CTX          FILTER_OTHER
+#define FILTER_TYPE_TRACE_WAKE         FILTER_OTHER
+#define FILTER_TYPE_TRACE_STACK                FILTER_OTHER
+#define FILTER_TYPE_TRACE_USER_STACK   FILTER_OTHER
+#define FILTER_TYPE_TRACE_BPRINT       FILTER_OTHER
+#define FILTER_TYPE_TRACE_PRINT                FILTER_OTHER
+#define FILTER_TYPE_TRACE_MMIO_RW      FILTER_OTHER
+#define FILTER_TYPE_TRACE_MMIO_MAP     FILTER_OTHER
+#define FILTER_TYPE_TRACE_BRANCH       FILTER_OTHER
+#define FILTER_TYPE(arg)               FILTER_TYPE_##arg
+
 #undef FTRACE_ENTRY
 #define FTRACE_ENTRY(name, struct_name, id, tstruct, print)		\
 int									\
@@ -123,6 +137,7 @@ ftrace_define_fields_##name(struct ftrace_event_call *event_call)	\
 {									\
 	struct struct_name field;					\
 	int ret;							\
+	int filter_type = FILTER_TYPE(id);				\
 									\
 	tstruct;							\
 									\
maintainer
#36 lore raw

Re: [PATCH 8/9] ftrace, perf: Add FILTER_TRACE_FN event field type

Quoted 5 lines · Jiri Olsa
Quoted 2 lines · Steven Rostedt

BTW, Please Cc Peter Zijlstra too, as he maintains perf inside the kernel.

thanks! ommited by mistake..

I hope not trimming the email was a mistake too? :-)

author Acked-by
#37 lore raw

Re: [PATCH 8/9] ftrace, perf: Add FILTER_TRACE_FN event field type

Quoted 8 lines · Peter Zijlstra
Quoted 5 lines · Jiri Olsa
Quoted 2 lines · Steven Rostedt

BTW, Please Cc Peter Zijlstra too, as he maintains perf inside the kernel.

thanks! ommited by mistake..

I hope not trimming the email was a mistake too? :-)

author
#38 lore raw

PATCH 9/9 ftrace, perf: Add filter support for function trace event

In reply to Jiri Olsa

Adding support to filter function trace event via perf interface. It is now possible to use filter interface in the perf tool like:

  perf record -e ftrace:function --filter="(ip == mm_*)" ls

The filter syntax is restricted to the the 'ip' field only, and following operators are accepted '==' '!=' '&&', ending up with the filter strings like:

"ip == f1 f2 ..." && "ip != f3 f4 ..." ...

The '==' operator adds trace filter with same efect as would be added via set_ftrace_filter file.

The '!=' operator adds trace filter with same efect as would be added via set_ftrace_notrace file.

The right side of the '!=', '==' operators is list of functions or regexp. to be added to filter separated by space. Same syntax is supported/required as for the set_ftrace_filter and set_ftrace_notrace files.

The '&&' operator is used for connecting multiple filter definitions together. It is possible to have more than one '==' and '!=' opearators within one filter string.

---
 kernel/trace/trace.h               |    4 +-
 kernel/trace/trace_events_filter.c |  111 +++++++++++++++++++++++++++++++++---
 kernel/trace/trace_export.c        |    5 ++
 3 files changed, 110 insertions(+), 10 deletions(-)
kernel/trace/trace.h +2 -2
index c4330dc..fde4d2a 100644
@@ -589,6 +589,8 @@ static inline int ftrace_trace_task(struct task_struct *task)
 static inline int ftrace_is_dead(void) { return 0; }
 #endif
 
+int ftrace_event_is_function(struct ftrace_event_call *call);
+
 /*
  * struct trace_parser - servers for reading the user input separated by spaces
  * @cont: set if the input is not complete - no final space char was found
@@ -765,9 +767,7 @@ struct filter_pred {
 	u64 			val;
 	struct regex		regex;
 	unsigned short		*ops;
-#ifdef CONFIG_FTRACE_STARTUP_TEST
 	struct ftrace_event_field *field;
-#endif
 	int 			offset;
 	int 			not;
 	int 			op;
kernel/trace/trace_events_filter.c +103 -8

171 lines not on this page · raw message

kernel/trace/trace_export.c +5 -0
index 7b035ab..46c35e2 100644
@@ -208,4 +208,9 @@ struct ftrace_event_call __used event_##call = {			\
 struct ftrace_event_call __used						\
 __attribute__((section("_ftrace_events"))) *__event_##call = &event_##call;
 
+int ftrace_event_is_function(struct ftrace_event_call *call)
+{
+	return call == &event_function;
+}
+
 #include "trace_entries.h"
Signature · 1 line
1.7.1
maintainer
#39 lore raw

Re: [PATCH 9/9] ftrace, perf: Add filter support for function trace event

Quoted 6 lines · Jiri Olsa

Adding support to filter function trace event via perf interface. It is now possible to use filter interface in the perf tool like:

  perf record -e ftrace:function --filter="(ip == mm_*)" ls
Quoted 7 lines

The filter syntax is restricted to the the 'ip' field only, and following operators are accepted '==' '!=' '&&', ending up with the filter strings like:

"ip == f1 f2 ..." && "ip != f3 f4 ..." ...

The '==' operator adds trace filter with same efect as would

effect

Quoted 3 lines

be added via set_ftrace_filter file.

The '!=' operator adds trace filter with same efect as would

effect

Quoted 10 lines

be added via set_ftrace_notrace file.

The right side of the '!=', '==' operators is list of functions or regexp. to be added to filter separated by space. Same syntax is supported/required as for the set_ftrace_filter and set_ftrace_notrace files.

The '&&' operator is used for connecting multiple filter definitions together. It is possible to have more than one '==' and '!=' opearators within one filter string.

operators

Interesting way to handle it. I'll have to test it out. I wonder if we could also make this work the same way with ftrace too.

-- Steve

Quoted 229 lines
---
 kernel/trace/trace.h               |    4 +-
 kernel/trace/trace_events_filter.c |  111 +++++++++++++++++++++++++++++++++---
 kernel/trace/trace_export.c        |    5 ++
 3 files changed, 110 insertions(+), 10 deletions(-)
kernel/trace/trace.h +2 -2
index c4330dc..fde4d2a 100644
@@ -589,6 +589,8 @@ static inline int ftrace_trace_task(struct task_struct *task)
 static inline int ftrace_is_dead(void) { return 0; }
 #endif
 
+int ftrace_event_is_function(struct ftrace_event_call *call);
+
 /*
  * struct trace_parser - servers for reading the user input separated by spaces
  * @cont: set if the input is not complete - no final space char was found
@@ -765,9 +767,7 @@ struct filter_pred {
 	u64 			val;
 	struct regex		regex;
 	unsigned short		*ops;
-#ifdef CONFIG_FTRACE_STARTUP_TEST
 	struct ftrace_event_field *field;
-#endif
 	int 			offset;
 	int 			not;
 	int 			op;
kernel/trace/trace_events_filter.c +103 -8
index 7b0b04c..7434f50 100644
@@ -54,6 +54,13 @@ struct filter_op {
 	int precedence;
 };
 
+static struct filter_op filter_ftrace_ops[] = {
+	{ OP_AND,	"&&",		1 },
+	{ OP_NE,	"!=",		2 },
+	{ OP_EQ,	"==",		2 },
+	{ OP_NONE,	"OP_NONE",	0 },
+};
+
 static struct filter_op filter_ops[] = {
 	{ OP_OR,	"||",		1 },
 	{ OP_AND,	"&&",		2 },
@@ -81,6 +88,7 @@ enum {
 	FILT_ERR_TOO_MANY_PREDS,
 	FILT_ERR_MISSING_FIELD,
 	FILT_ERR_INVALID_FILTER,
+	FILT_ERR_IP_FIELD_ONLY,
 };
 
 static char *err_text[] = {
@@ -96,6 +104,7 @@ static char *err_text[] = {
 	"Too many terms in predicate expression",
 	"Missing field name and/or value",
 	"Meaningless filter expression",
+	"Only 'ip' field is supported for function trace",
 };
 
 struct opstack_op {
@@ -992,7 +1001,12 @@ static int init_pred(struct filter_parse_state *ps,
 			fn = filter_pred_strloc;
 		else
 			fn = filter_pred_pchar;
-	} else if (!is_function_field(field)) {
+	} else if (is_function_field(field)) {
+		if (strcmp(field->name, "ip")) {
+			parse_error(ps, FILT_ERR_IP_FIELD_ONLY, 0);
+			return -EINVAL;
+		}
+	} else {
 		if (field->is_signed)
 			ret = strict_strtoll(pred->regex.pattern, 0, &val);
 		else
@@ -1339,10 +1353,8 @@ static struct filter_pred *create_pred(struct filter_parse_state *ps,
 
 	strcpy(pred.regex.pattern, operand2);
 	pred.regex.len = strlen(pred.regex.pattern);
-
-#ifdef CONFIG_FTRACE_STARTUP_TEST
 	pred.field = field;
-#endif
+
 	return init_pred(ps, field, &pred) ? NULL : &pred;
 }
 
@@ -1894,6 +1906,81 @@ void ftrace_profile_free_filter(struct perf_event *event)
 	__free_filter(filter);
 }
 
+struct function_filter_data {
+	struct ftrace_ops *ops;
+	int first_filter;
+	int first_notrace;
+};
+
+static int __ftrace_function_set_filter(int filter, char *buf, int len,
+					struct function_filter_data *data)
+{
+	int *reset;
+
+	reset = filter ? &data->first_filter : &data->first_notrace;
+
+	if (filter)
+		ftrace_set_filter(data->ops, buf, len, *reset);
+	else
+		ftrace_set_notrace(data->ops, buf, len, *reset);
+
+	if (*reset)
+		*reset = 0;
+
+	return WALK_PRED_DEFAULT;
+}
+
+static int ftrace_function_check_pred(struct filter_pred *pred)
+{
+	struct ftrace_event_field *field = pred->field;
+
+	/*
+	  Check the predicate for function trace, verify:
+	   - only '==' and '!=' is used
+	   - the 'ip' field is used
+	*/
+	if (WARN((pred->op != OP_EQ) && (pred->op != OP_NE),
+		 "wrong operator for function filter: %d\n", pred->op))
+		return -EINVAL;
+
+	if (strcmp(field->name, "ip"))
+		return -EINVAL;
+
+	return 0;
+}
+
+static int ftrace_function_set_filter_cb(enum move_type move,
+					 struct filter_pred *pred,
+					 int *err, void *data)
+{
+	if ((move != MOVE_DOWN) ||
+	    (pred->left != FILTER_PRED_INVALID))
+		return WALK_PRED_DEFAULT;
+
+	/* Double checking the predicate is valid for function trace. */
+	*err = ftrace_function_check_pred(pred);
+	if (*err)
+		return WALK_PRED_ABORT;
+
+	return __ftrace_function_set_filter(pred->op == OP_EQ,
+					    pred->regex.pattern,
+					    pred->regex.len,
+					    data);
+}
+
+static int ftrace_function_set_filter(struct perf_event *event,
+				      struct event_filter *filter)
+{
+	struct function_filter_data data = {
+		.first_filter  = 1,
+		.first_notrace = 1,
+		.ops           = &event->ftrace_ops,
+	};
+
+	return walk_pred_tree(filter->preds, filter->root,
+			      ftrace_function_set_filter_cb, &data);
+}
+
 int ftrace_profile_set_filter(struct perf_event *event, int event_id,
 			      char *filter_str)
 {
@@ -1901,6 +1988,7 @@ int ftrace_profile_set_filter(struct perf_event *event, int event_id,
 	struct event_filter *filter;
 	struct filter_parse_state *ps;
 	struct ftrace_event_call *call;
+	struct filter_op *fops = filter_ops;
 
 	mutex_lock(&event_mutex);
 
@@ -1925,14 +2013,21 @@ int ftrace_profile_set_filter(struct perf_event *event, int event_id,
 	if (!ps)
 		goto free_filter;
 
-	parse_init(ps, filter_ops, filter_str);
+	if (ftrace_event_is_function(call))
+		fops = filter_ftrace_ops;
+
+	parse_init(ps, fops, filter_str);
 	err = filter_parse(ps);
 	if (err)
 		goto free_ps;
 
 	err = replace_preds(call, filter, ps, filter_str, false);
-	if (!err)
-		event->filter = filter;
+	if (!err) {
+		if (ftrace_event_is_function(call))
+			err = ftrace_function_set_filter(event, filter);
+		else
+			event->filter = filter;
+	}
 
 free_ps:
 	filter_opstack_clear(ps);
@@ -1940,7 +2035,7 @@ free_ps:
 	kfree(ps);
 
 free_filter:
-	if (err)
+	if (err || ftrace_event_is_function(call))
 		__free_filter(filter);
 
 out_unlock:
kernel/trace/trace_export.c +5 -0
index 7b035ab..46c35e2 100644
@@ -208,4 +208,9 @@ struct ftrace_event_call __used event_##call = {			\
 struct ftrace_event_call __used						\
 __attribute__((section("_ftrace_events"))) *__event_##call = &event_##call;
 
+int ftrace_event_is_function(struct ftrace_event_call *call)
+{
+	return call == &event_function;
+}
+
 #include "trace_entries.h"
author
#40 lore raw
In reply to Jiri Olsa

hi,
here's another version of perf support for function trace with filter. The changeset is working and hopefully is not introducing more bugs.. ;) still testing though..

It's still marked RFC since I'm not sure there's better way to go with patches 2 and 10. Also not sure if the condition fixed by patch 7 was not intentional. The rest is fixed/updated version of the v1 changes.

attached patches:
01/10 ftrace: Fix possible NULL dereferencing in __ftrace_hash_rec_update 02/10 ftrace: Change mcount call replacement logic
03/10 ftrace: Add enable/disable ftrace_ops control interface 04/10 ftrace, perf: Add open/close tracepoint perf registration actions 05/10 ftrace, perf: Add add/del tracepoint perf registration actions 06/10 ftrace, perf: Add support to use function tracepoint in perf 07/10 ftrace: Change filter/notrace set functions to return exit code 08/10 ftrace, perf: Distinguish ftrace function event field type 09/10 ftrace, perf: Add filter support for function trace event 10/10 ftrace, graph: Add global_ops filter callback for graph tracing

v2 changes:
01/10 - keeping the old fix instead of adding hash_has_contents func
I'll send separating patchset for this
02/10 - using different way to avoid the issue (3/9 in v1) 03/10 - using the way proposed by Steven for controling ftrace_ops

         (4/9 in v1)
 06/10 - added check ensuring the ftrace:function event could be used by
         root only (7/9 in v1)

08/10 - added more description (8/9 in v1)
09/10 - changed '&&' operator to '||' which seems more suitable
in this case (9/9 in v1)

thanks for comments,
jirka

---
 include/linux/ftrace.h             |   18 +++-
 include/linux/ftrace_event.h       |    9 +-
 include/linux/perf_event.h         |    3 +
 kernel/trace/ftrace.c              |  202 +++++++++++++++++++++++++++++------
 kernel/trace/trace.h               |   11 ++-
 kernel/trace/trace_event_perf.c    |  212 +++++++++++++++++++++++++++++-------
 kernel/trace/trace_events.c        |   12 ++-
 kernel/trace/trace_events_filter.c |  116 ++++++++++++++++++-
 kernel/trace/trace_export.c        |   53 ++++++++-
 kernel/trace/trace_kprobe.c        |    8 +-
 kernel/trace/trace_syscalls.c      |   18 +++-
 11 files changed, 562 insertions(+), 100 deletions(-)
author
#41 lore raw

PATCH v2 1/10 ftrace: Fix possible NULL dereferencing in __ftrace_hash_rec_update

We need to check the existence of the other_hash before we touch its count variable.

This issue is hit only when non global ftrace_ops is used. The global ftrace_ops is initialized with empty hashes.

---
 kernel/trace/ftrace.c |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)
kernel/trace/ftrace.c +2 -1
index b1e8943..c6d0293 100644
@@ -1372,7 +1372,8 @@ static void __ftrace_hash_rec_update(struct ftrace_ops *ops,
 			if (filter_hash && in_hash && !in_other_hash)
 				match = 1;
 			else if (!filter_hash && in_hash &&
-				 (in_other_hash || !other_hash->count))
+				 (in_other_hash ||
+				  !other_hash || !other_hash->count))
 				match = 1;
 		}
 		if (!match)
Signature · 1 line
1.7.1
author
#42 lore raw

PATCH v2 2/10 ftrace: Change mcount call replacement logic

In reply to Jiri Olsa

The current logic of updating calls is to do the mcount replacement only when ftrace_ops is being registered. When ftrace_ops is being unregistered then only in case it was the last registered ftrace_ops, all calls are disabled.

This is an issue when ftrace_ops without FTRACE_OPS_FL_GLOBAL flag is being unregistered, because all the functions stays enabled and thus inherrited by global_ops, like in following scenario:

- set restricting global filter
- enable function trace
- register/unregister ftrace_ops with flags != FTRACE_OPS_FL_GLOBAL and with no filter

Now the global_ops will get by all the functions regardless the global_ops filter. So we need all functions that where enabled via this ftrace_ops and are not part of global filter to be disabled.

Note, currently if there are only global ftrace_ops registered, there's no filter hash check and the filter is represented only by enabled records.

Changing the ftrace_shutdown logic to ensure the replacement is called for each ftrace_ops being unregistered.

Also changing FTRACE_ENABLE_CALLS into FTRACE_UPDATE_CALLS calls which seems more suitable now.

---
 kernel/trace/ftrace.c |   27 +++++++++++++--------------
 1 files changed, 13 insertions(+), 14 deletions(-)
kernel/trace/ftrace.c +13 -14
index c6d0293..b79ab33 100644
@@ -948,7 +948,7 @@ struct ftrace_func_probe {
 };
 
 enum {
-	FTRACE_ENABLE_CALLS		= (1 << 0),
+	FTRACE_UPDATE_CALLS		= (1 << 0),
 	FTRACE_DISABLE_CALLS		= (1 << 1),
 	FTRACE_UPDATE_TRACE_FUNC	= (1 << 2),
 	FTRACE_START_FUNC_RET		= (1 << 3),
@@ -1520,7 +1520,7 @@ int ftrace_text_reserved(void *start, void *end)
 
 
 static int
-__ftrace_replace_code(struct dyn_ftrace *rec, int enable)
+__ftrace_replace_code(struct dyn_ftrace *rec, int update)
 {
 	unsigned long ftrace_addr;
 	unsigned long flag = 0UL;
@@ -1528,17 +1528,17 @@ __ftrace_replace_code(struct dyn_ftrace *rec, int enable)
 	ftrace_addr = (unsigned long)FTRACE_ADDR;
 
 	/*
-	 * If we are enabling tracing:
+	 * If we are updating calls:
 	 *
 	 *   If the record has a ref count, then we need to enable it
 	 *   because someone is using it.
 	 *
 	 *   Otherwise we make sure its disabled.
 	 *
-	 * If we are disabling tracing, then disable all records that
+	 * If we are disabling calls, then disable all records that
 	 * are enabled.
 	 */
-	if (enable && (rec->flags & ~FTRACE_FL_MASK))
+	if (update && (rec->flags & ~FTRACE_FL_MASK))
 		flag = FTRACE_FL_ENABLED;
 
 	/* If the state of this record hasn't changed, then do nothing */
@@ -1554,7 +1554,7 @@ __ftrace_replace_code(struct dyn_ftrace *rec, int enable)
 	return ftrace_make_nop(NULL, rec, ftrace_addr);
 }
 
-static void ftrace_replace_code(int enable)
+static void ftrace_replace_code(int update)
 {
 	struct dyn_ftrace *rec;
 	struct ftrace_page *pg;
@@ -1568,7 +1568,7 @@ static void ftrace_replace_code(int enable)
 		if (rec->flags & FTRACE_FL_FREE)
 			continue;
 
-		failed = __ftrace_replace_code(rec, enable);
+		failed = __ftrace_replace_code(rec, update);
 		if (failed) {
 			ftrace_bug(failed, rec->ip);
 			/* Stop processing */
@@ -1624,7 +1624,7 @@ static int __ftrace_modify_code(void *data)
 	 */
 	function_trace_stop++;
 
-	if (*command & FTRACE_ENABLE_CALLS)
+	if (*command & FTRACE_UPDATE_CALLS)
 		ftrace_replace_code(1);
 	else if (*command & FTRACE_DISABLE_CALLS)
 		ftrace_replace_code(0);
@@ -1692,7 +1692,7 @@ static int ftrace_startup(struct ftrace_ops *ops, int command)
 		return -ENODEV;
 
 	ftrace_start_up++;
-	command |= FTRACE_ENABLE_CALLS;
+	command |= FTRACE_UPDATE_CALLS;
 
 	/* ops marked global share the filter hashes */
 	if (ops->flags & FTRACE_OPS_FL_GLOBAL) {
@@ -1744,8 +1744,7 @@ static void ftrace_shutdown(struct ftrace_ops *ops, int command)
 	if (ops != &global_ops || !global_start_up)
 		ops->flags &= ~FTRACE_OPS_FL_ENABLED;
 
-	if (!ftrace_start_up)
-		command |= FTRACE_DISABLE_CALLS;
+	command |= FTRACE_UPDATE_CALLS;
 
 	if (saved_ftrace_func != ftrace_trace_function) {
 		saved_ftrace_func = ftrace_trace_function;
@@ -1767,7 +1766,7 @@ static void ftrace_startup_sysctl(void)
 	saved_ftrace_func = NULL;
 	/* ftrace_start_up is true if we want ftrace running */
 	if (ftrace_start_up)
-		ftrace_run_update_code(FTRACE_ENABLE_CALLS);
+		ftrace_run_update_code(FTRACE_UPDATE_CALLS);
 }
 
 static void ftrace_shutdown_sysctl(void)
@@ -2920,7 +2919,7 @@ ftrace_set_regex(struct ftrace_ops *ops, unsigned char *buf, int len,
 	ret = ftrace_hash_move(ops, enable, orig_hash, hash);
 	if (!ret && ops->flags & FTRACE_OPS_FL_ENABLED
 	    && ftrace_enabled)
-		ftrace_run_update_code(FTRACE_ENABLE_CALLS);
+		ftrace_run_update_code(FTRACE_UPDATE_CALLS);
 
 	mutex_unlock(&ftrace_lock);
 
@@ -3108,7 +3107,7 @@ ftrace_regex_release(struct inode *inode, struct file *file)
 				       orig_hash, iter->hash);
 		if (!ret && (iter->ops->flags & FTRACE_OPS_FL_ENABLED)
 		    && ftrace_enabled)
-			ftrace_run_update_code(FTRACE_ENABLE_CALLS);
+			ftrace_run_update_code(FTRACE_UPDATE_CALLS);
 
 		mutex_unlock(&ftrace_lock);
 	}
Signature · 1 line
1.7.1
maintainer
#43 lore raw

Re: [PATCHv2 02/10] ftrace: Change mcount call replacement logic

Quoted 5 lines · Jiri Olsa

The current logic of updating calls is to do the mcount replacement only when ftrace_ops is being registered. When ftrace_ops is being unregistered then only in case it was the last registered ftrace_ops, all calls are disabled.

Quoted 8 lines

This is an issue when ftrace_ops without FTRACE_OPS_FL_GLOBAL flag is being unregistered, because all the functions stays enabled and thus inherrited by global_ops, like in following scenario:

- set restricting global filter
- enable function trace
- register/unregister ftrace_ops with flags != FTRACE_OPS_FL_GLOBAL and with no filter

I don't see this problem. I just changed stack_tracer to have its own filter (I've been wanting to do that for a long time, so when I saw this email, I decided it's a good time to implement it).

Here's what I did:

# echo schedule > set_ftrace_filter
# cat set_ftrace_filter
schedule
# cat enabled_functions
schedule (1)
# echo 1 > /proc/sys/kernel/stack_tracer_enabled
# cat enabled_functions
do_one_initcall (1)
match_dev_by_uuid (1)
name_to_dev_t (1)
idle_notifier_unregister (1)
idle_notifier_register (1)
start_thread_common.constprop.6 (1)
enter_idle (1)
exit_idle (1)
cpu_idle (1)
__show_regs (1)
release_thread (1)
[...]
_cond_resched (1)
preempt_schedule_irq (1)
schedule (2)
io_schedule (1)
yield_to (1)
yield (1)

// note that schedule is (2)

# echo 0 > /proc/sys/kernel/stack_tracer_enabled
# cat enabled_functions
schedule (1)
Quoted 4 lines

Now the global_ops will get by all the functions regardless the global_ops filter. So we need all functions that where enabled via this ftrace_ops and are not part of global filter to be disabled.

The global functions are not at issue here. What do you see?

Maybe I fixed something as I'm using the latest tip/perf/core. Note, I can send you the stack_tracer patch if you want to take a look at this example. I need to clean it up too.

Quoted 10 lines

Note, currently if there are only global ftrace_ops registered, there's no filter hash check and the filter is represented only by enabled records.

Changing the ftrace_shutdown logic to ensure the replacement is called for each ftrace_ops being unregistered.

Also changing FTRACE_ENABLE_CALLS into FTRACE_UPDATE_CALLS calls which seems more suitable now.

I still think this patch is wrong. What's the problem you are seeing?

-- Steve

author
#44 lore raw

Re: [PATCHv2 02/10] ftrace: Change mcount call replacement logic

Quoted 59 lines · Steven Rostedt
Quoted 5 lines · Jiri Olsa

The current logic of updating calls is to do the mcount replacement only when ftrace_ops is being registered. When ftrace_ops is being unregistered then only in case it was the last registered ftrace_ops, all calls are disabled.

Quoted 8 lines

This is an issue when ftrace_ops without FTRACE_OPS_FL_GLOBAL flag is being unregistered, because all the functions stays enabled and thus inherrited by global_ops, like in following scenario:

- set restricting global filter
- enable function trace
- register/unregister ftrace_ops with flags != FTRACE_OPS_FL_GLOBAL and with no filter

I don't see this problem. I just changed stack_tracer to have its own filter (I've been wanting to do that for a long time, so when I saw this email, I decided it's a good time to implement it).

Here's what I did:

# echo schedule > set_ftrace_filter
# cat set_ftrace_filter
schedule
# cat enabled_functions
schedule (1)
# echo 1 > /proc/sys/kernel/stack_tracer_enabled
# cat enabled_functions
do_one_initcall (1)
match_dev_by_uuid (1)
name_to_dev_t (1)
idle_notifier_unregister (1)
idle_notifier_register (1)
start_thread_common.constprop.6 (1)
enter_idle (1)
exit_idle (1)
cpu_idle (1)
__show_regs (1)
release_thread (1)
[...]
_cond_resched (1)
preempt_schedule_irq (1)
schedule (2)
io_schedule (1)
yield_to (1)
yield (1)

// note that schedule is (2)

# echo 0 > /proc/sys/kernel/stack_tracer_enabled
# cat enabled_functions
schedule (1)
Quoted 4 lines

Now the global_ops will get by all the functions regardless the global_ops filter. So we need all functions that where enabled via this ftrace_ops and are not part of global filter to be disabled.

Quoted 5 lines

The global functions are not at issue here. What do you see?

Maybe I fixed something as I'm using the latest tip/perf/core. Note, I can send you the stack_tracer patch if you want to take a look at this example. I need to clean it up too.

that would be great, thanks

Quoted 6 lines
Quoted 6 lines

Note, currently if there are only global ftrace_ops registered, there's no filter hash check and the filter is represented only by enabled records.

Quoted 7 lines
Quoted 5 lines

Changing the ftrace_shutdown logic to ensure the replacement is called for each ftrace_ops being unregistered.

Also changing FTRACE_ENABLE_CALLS into FTRACE_UPDATE_CALLS calls which seems more suitable now.

I still think this patch is wrong. What's the problem you are seeing?

let me try with an example..

say we have only 2 traceable functions - A and B ;)

1) set global filter for function A with 'echo A > ./set_ftrace_filter' a - A is put to the global_ops filter

2) enable function trace with 'echo function > current_tracer'
a - register_ftrace_function is called with function trace ftrace_ops (GLOBAL flag) b - update_ftrace_function is called, setting ftrace_ops callback function
to be called directly from the assembly entry_* code
c - ftrace_hash_rec_enable is called, and dyn_ftrace record
for function A is updated:
A::flags|FL_MASK = 1
d - ftrace_replace_code(1) is called, and function A is
brought in to life

3) enable function trace via perf ftrace_ops
a - register_ftrace_function is called with perf event ftrace_ops (!GLOBAL flag) b - update_ftrace_function is called, setting ftrace_ops_list_func
function to be called from the assembly entry_* code and handle the ftrace_ops' dispatch
c - ftrace_hash_rec_enable is called, and A and B dyn_ftrace records
are updated:
A::flags|FL_MASK = 2
B::flags|FL_MASK = 1
d - ftrace_replace_code(1) is called, and function B is
brought in to life

4) disable function trace via perf ftrace_ops
a - unregister_ftrace_function is called with perf event ftrace_ops (same as in step 3) b - update_ftrace_function is called, setting global ftrace_ops (from step 2)
callback function to be called directly from the assembly entry_* code
c - ftrace_hash_rec_disable is called, and A and B dyn_ftrace
records are updated:
A::flags|FL_MASK = 1
B::flags|FL_MASK = 0
d - ??? see below..

Now, only the global function trace ftrace_ops is enabled (from step 2), but both A and B are alive and feeding the tracer despite its filter (function A), because its ftrace_ops is directly linked to the assembly entry_* code (step 4b).

The reason is that even though we updated the B's dyn_ftrace record (step 4c) to be 'B::flags|FL_MASK == 0', we did not update the B call itself and disable it.

If we'd call ftrace_replace_code(1) at 4d), the B function would be disabled, and we would get expected behaviour.

So thats the reason I think we should update the calls (mcount call code) each time we unregister the ftrace_ops, because some records could be enabled only for the specific ftrace_ops and we need to put them down when we disable this ftrace_ops.

hopefully it make any sense.. :)

thanks,
jirka

maintainer
#45 lore raw

Re: [PATCHv2 02/10] ftrace: Change mcount call replacement logic

Quoted 49 lines · Jiri Olsa

let me try with an example..

say we have only 2 traceable functions - A and B ;)

1) set global filter for function A with 'echo A > ./set_ftrace_filter' a - A is put to the global_ops filter

2) enable function trace with 'echo function > current_tracer'
a - register_ftrace_function is called with function trace ftrace_ops (GLOBAL flag) b - update_ftrace_function is called, setting ftrace_ops callback function
to be called directly from the assembly entry_* code
c - ftrace_hash_rec_enable is called, and dyn_ftrace record
for function A is updated:
A::flags|FL_MASK = 1
d - ftrace_replace_code(1) is called, and function A is
brought in to life

3) enable function trace via perf ftrace_ops
a - register_ftrace_function is called with perf event ftrace_ops (!GLOBAL flag) b - update_ftrace_function is called, setting ftrace_ops_list_func
function to be called from the assembly entry_* code and handle the ftrace_ops' dispatch
c - ftrace_hash_rec_enable is called, and A and B dyn_ftrace records
are updated:
A::flags|FL_MASK = 2
B::flags|FL_MASK = 1
d - ftrace_replace_code(1) is called, and function B is
brought in to life

4) disable function trace via perf ftrace_ops
a - unregister_ftrace_function is called with perf event ftrace_ops (same as in step 3) b - update_ftrace_function is called, setting global ftrace_ops (from step 2)
callback function to be called directly from the assembly entry_* code
c - ftrace_hash_rec_disable is called, and A and B dyn_ftrace
records are updated:
A::flags|FL_MASK = 1
B::flags|FL_MASK = 0
d - ??? see below..

Now, only the global function trace ftrace_ops is enabled (from step 2), but both A and B are alive and feeding the tracer despite its filter (function A), because its ftrace_ops is directly linked to the assembly entry_* code (step 4b).

The reason is that even though we updated the B's dyn_ftrace record (step 4c) to be 'B::flags|FL_MASK == 0', we did not update the B call itself and disable it.

If we'd call ftrace_replace_code(1) at 4d), the B function would be disabled, and we would get expected behaviour.

Quoted 7 lines

So thats the reason I think we should update the calls (mcount call code) each time we unregister the ftrace_ops, because some records could be enabled only for the specific ftrace_ops and we need to put them down when we disable this ftrace_ops.

hopefully it make any sense.. :)

Ah, OK, I am able to trigger this. I see what you mean. Yeah, now that we have a counter, it should enable/disable based on the counter. And the DISABLE is still needed for just the "disable regardless"
(/proc/sys/kernel/ftrace_enabled = 0).

I'll apply your patch and see how it works. Thanks!

-- Steve

maintainer
#46 lore raw

Re: [PATCHv2 02/10] ftrace: Change mcount call replacement logic

In reply to Jiri Olsa
Quoted 6 lines · Jiri Olsa

The current logic of updating calls is to do the mcount replacement only when ftrace_ops is being registered. When ftrace_ops is being unregistered then only in case it was the last registered ftrace_ops, all calls are disabled.

This is an issue when ftrace_ops without FTRACE_OPS_FL_GLOBAL flag

Actually it has nothing to do with the global flag. The bug is just two different ftrace_ops. As I can make the problem go either way (by disabling the global_tracer I screwed up the stack_tracer). I was also able to screw up the stack tracer with the function_profile_ops that does not have the flag set either.

I was able to trigger this bug with just using the new stack_trace ops and the function profiling ops. Neither of them have the global flag set.

Quoted 7 lines

is being unregistered, because all the functions stays enabled and thus inherrited by global_ops, like in following scenario:

- set restricting global filter
- enable function trace
- register/unregister ftrace_ops with flags != FTRACE_OPS_FL_GLOBAL and with no filter

The real problem is:

- set restricting filter on one ftrace_ops
- enable function tracing for that ftrace_ops
- register/unregister another ftrace_ops with no filter
- Now the first ftrace_ops has all the functions of the last

Quoted 8 lines

Now the global_ops will get by all the functions regardless the global_ops filter. So we need all functions that where enabled via this ftrace_ops and are not part of global filter to be disabled.

Note, currently if there are only global ftrace_ops registered, there's no filter hash check and the filter is represented only by enabled records.

This isn't totally true either, as function_profile_call has the same issue. This bug does exist in mainline today. If you unconfigure function graph tracing, then the function profiling uses function tracer instead of function graph tracer to profile.

Here, I just reproduced it:

 # cd /debug/tracing/
 # echo schedule > set_ftrace_filter
 # echo 1 > function_profile_enabled
 # echo 0 > function_profile_enabled
 # cat set_ftrace_filter
schedule
 # cat trace
# tracer: function
#
# entries-in-buffer/entries-written: 87869/87869   #P:4
#
#                              _-----=> irqs-off
#                             / _----=> need-resched
#                            | / _---=> hardirq/softirq
#                            || / _--=> preempt-depth
#                            ||| /     delay
#           TASK-PID   CPU#  ||||    TIMESTAMP  FUNCTION
#              | |       |   ||||       |         |
     kworker/1:1-49    [001] d..2    13.258661: __rcu_read_unlock <-__schedule
     kworker/1:1-49    [001] d..2    13.258661: _raw_spin_lock <-__schedule
     kworker/1:1-49    [001] d..2    13.258661: add_preempt_count <-_raw_spin_lock
     kworker/1:1-49    [001] d..3    13.258662: put_prev_task_fair <-__schedule
     kworker/1:1-49    [001] d..3    13.258662: pick_next_task_fair <-pick_next_task
     kworker/1:1-49    [001] d..3    13.258662: pick_next_task_stop <-pick_next_task
     kworker/1:1-49    [001] d..3    13.258663: pick_next_task_rt <-pick_next_task
     kworker/1:1-49    [001] d..3    13.258663: pick_next_task_fair <-pick_next_task
     kworker/1:1-49    [001] d..3    13.258663: pick_next_task_idle <-pick_next_task
     kworker/1:1-49    [001] d..3    13.258664: sched_switch: prev_comm=kworker/1:1 prev_pid=49 prev_prio=120 prev_state=S ==> next_comm=kworker/0:0 next_pid=0 next_prio=120
     kworker/1:1-49    [001] d..3    13.258664: __switch_to <-__schedule
          <idle>-0     [001] ...1    13.258665: __schedule <-schedule
          <idle>-0     [001] ...1    13.258665: add_preempt_count <-__schedule
          <idle>-0     [001] ...2    13.258665: rcu_note_context_switch <-__schedule
          <idle>-0     [001] ...2    13.258665: rcu_utilization: Start context switch

I'll put this to the front of my patch queue and also add it to urgent. I'll also rewrite your change log to remove the reference to the global ops.

-- Steve

Quoted 9 lines

Changing the ftrace_shutdown logic to ensure the replacement is called for each ftrace_ops being unregistered.

Also changing FTRACE_ENABLE_CALLS into FTRACE_UPDATE_CALLS calls which seems more suitable now.

---

author
#47 lore raw

Re: [PATCHv2 02/10] ftrace: Change mcount call replacement logic

Quoted 1 line · Steven Rostedt

On Mon, 2011-12-05 at 18:22 +0100, Jiri Olsa wrote:

SNIP

Quoted 7 lines
          <idle>-0     [001] ...1    13.258665: __schedule <-schedule
          <idle>-0     [001] ...1    13.258665: add_preempt_count <-__schedule
          <idle>-0     [001] ...2    13.258665: rcu_note_context_switch <-__schedule
          <idle>-0     [001] ...2    13.258665: rcu_utilization: Start context switch

I'll put this to the front of my patch queue and also add it to urgent. I'll also rewrite your change log to remove the reference to the global

ok, I'll send new version without this one

thanks,
jirka

maintainer
#48 lore raw

Re: [PATCHv2 02/10] ftrace: Change mcount call replacement logic

Quoted 10 lines · Jiri Olsa
Quoted 1 line · Steven Rostedt

On Mon, 2011-12-05 at 18:22 +0100, Jiri Olsa wrote:

SNIP

Quoted 5 lines
          <idle>-0     [001] ...1    13.258665: __schedule <-schedule
          <idle>-0     [001] ...1    13.258665: add_preempt_count <-__schedule
          <idle>-0     [001] ...2    13.258665: rcu_note_context_switch <-__schedule
          <idle>-0     [001] ...2    13.258665: rcu_utilization: Start context switch
Quoted 4 lines
Quoted 2 lines

I'll put this to the front of my patch queue and also add it to urgent. I'll also rewrite your change log to remove the reference to the global

ok, I'll send new version without this one

No need, I've already queued it and will be the first patch in the patchset I send out soon.

-- Steve

maintainer
#49 lore raw

Re: [PATCHv2 02/10] ftrace: Change mcount call replacement logic

Quoted 11 lines · Steven Rostedt
Quoted 10 lines · Jiri Olsa
Quoted 1 line · Steven Rostedt

On Mon, 2011-12-05 at 18:22 +0100, Jiri Olsa wrote:

SNIP

Quoted 5 lines
          <idle>-0     [001] ...1    13.258665: __schedule <-schedule
          <idle>-0     [001] ...1    13.258665: add_preempt_count <-__schedule
          <idle>-0     [001] ...2    13.258665: rcu_note_context_switch <-__schedule
          <idle>-0     [001] ...2    13.258665: rcu_utilization: Start context switch
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

I'll put this to the front of my patch queue and also add it to urgent. I'll also rewrite your change log to remove the reference to the global

ok, I'll send new version without this one

No need, I've already queued it and will be the first patch in the patchset I send out soon.

Or do you mean you'll send out a new patch series without this patch.

/me should not send email out when he first wakes up.

-- Steve

author
#50 lore raw

Re: [PATCHv2 02/10] ftrace: Change mcount call replacement logic

Quoted 15 lines · Steven Rostedt
Quoted 14 lines · Steven Rostedt
Quoted 13 lines · Jiri Olsa
Quoted 1 line · Steven Rostedt

On Mon, 2011-12-05 at 18:22 +0100, Jiri Olsa wrote:

SNIP

Quoted 7 lines
          <idle>-0     [001] ...1    13.258665: __schedule <-schedule
          <idle>-0     [001] ...1    13.258665: add_preempt_count <-__schedule
          <idle>-0     [001] ...2    13.258665: rcu_note_context_switch <-__schedule
          <idle>-0     [001] ...2    13.258665: rcu_utilization: Start context switch

I'll put this to the front of my patch queue and also add it to urgent. I'll also rewrite your change log to remove the reference to the global

Quoted 7 lines
Quoted 5 lines
Quoted 1 line

ok, I'll send new version without this one

No need, I've already queued it and will be the first patch in the patchset I send out soon.

Or do you mean you'll send out a new patch series without this patch.

yep :)

Quoted 6 lines

/me should not send email out when he first wakes up.

-- Steve

#51 lore raw

[tip:perf/core] ftrace: Fix unregister ftrace_ops accounting

In reply to Jiri Olsa
Commit-ID:  30fb6aa74011dcf595f306ca2727254d708b786e
Gitweb:     http://git.kernel.org/tip/30fb6aa74011dcf595f306ca2727254d708b786e
Author:     Jiri Olsa <[email protected]>
AuthorDate: Mon, 5 Dec 2011 18:22:48 +0100
Committer:  Steven Rostedt <[email protected]>
CommitDate: Wed, 21 Dec 2011 07:09:14 -0500

ftrace: Fix unregister ftrace_ops accounting

Multiple users of the function tracer can register their functions with the ftrace_ops structure. The accounting within ftrace will update the counter on each function record that is being traced. When the ftrace_ops filtering adds or removes functions, the function records will be updated accordingly if the ftrace_ops is still registered.

When a ftrace_ops is removed, the counter of the function records, that the ftrace_ops traces, are decremented. When they reach zero the functions that they represent are modified to stop calling the mcount code.

When changes are made, the code is updated via stop_machine() with a command passed to the function to tell it what to do. There is an ENABLE and DISABLE command that tells the called function to enable or disable the functions. But the ENABLE is really a misnomer as it should just update the records, as records that have been enabled and now have a count of zero should be disabled.

The DISABLE command is used to disable all functions regardless of their counter values. This is the big off switch and is not the complement of the ENABLE command.

To make matters worse, when a ftrace_ops is unregistered and there is another ftrace_ops registered, neither the DISABLE nor the ENABLE command are set when calling into the stop_machine() function and the records will not be updated to match their counter. A command is passed to that function that will update the mcount code to call the registered callback directly if it is the only one left. This means that the ftrace_ops that is still registered will have its callback called by all functions that have been set for it as well as the ftrace_ops that was just unregistered.

Here's a way to trigger this bug. Compile the kernel with CONFIG_FUNCTION_PROFILER set and with CONFIG_FUNCTION_GRAPH not set:

CONFIG_FUNCTION_PROFILER=y
# CONFIG_FUNCTION_GRAPH is not set

This will force the function profiler to use the function tracer instead of the function graph tracer.

  # cd /sys/kernel/debug/tracing
  # echo schedule > set_ftrace_filter
  # echo function > current_tracer
  # cat set_ftrace_filter
 schedule
  # cat trace
 # tracer: nop
 #
 # entries-in-buffer/entries-written: 692/68108025   #P:4
 #
 #                              _-----=> irqs-off
 #                             / _----=> need-resched
 #                            | / _---=> hardirq/softirq
 #                            || / _--=> preempt-depth
 #                            ||| /     delay
 #           TASK-PID   CPU#  ||||    TIMESTAMP  FUNCTION
 #              | |       |   ||||       |         |
      kworker/0:2-909   [000] ....   531.235574: schedule <-worker_thread
           <idle>-0     [001] .N..   531.235575: schedule <-cpu_idle
      kworker/0:2-909   [000] ....   531.235597: schedule <-worker_thread
             sshd-2563  [001] ....   531.235647: schedule <-schedule_hrtimeout_range_clock
  # echo 1 > function_profile_enabled
  # echo 0 > function_porfile_enabled
  # cat set_ftrace_filter
 schedule
  # cat trace
 # tracer: function
 #
 # entries-in-buffer/entries-written: 159701/118821262   #P:4
 #
 #                              _-----=> irqs-off
 #                             / _----=> need-resched
 #                            | / _---=> hardirq/softirq
 #                            || / _--=> preempt-depth
 #                            ||| /     delay
 #           TASK-PID   CPU#  ||||    TIMESTAMP  FUNCTION
 #              | |       |   ||||       |         |
           <idle>-0     [002] ...1   604.870655: local_touch_nmi <-cpu_idle
           <idle>-0     [002] d..1   604.870655: enter_idle <-cpu_idle
           <idle>-0     [002] d..1   604.870656: atomic_notifier_call_chain <-enter_idle
           <idle>-0     [002] d..1   604.870656: __atomic_notifier_call_chain <-atomic_notifier_call_chain

The same problem could have happened with the trace_probe_ops, but they are modified with the set_frace_filter file which does the update at closure of the file.

The simple solution is to change ENABLE to UPDATE and call it every time an ftrace_ops is unregistered.

---
 kernel/trace/ftrace.c |   27 +++++++++++++--------------
 1 files changed, 13 insertions(+), 14 deletions(-)
kernel/trace/ftrace.c +13 -14
index b1e8943..25b4f4d 100644
@@ -948,7 +948,7 @@ struct ftrace_func_probe {
 };
 
 enum {
-	FTRACE_ENABLE_CALLS		= (1 << 0),
+	FTRACE_UPDATE_CALLS		= (1 << 0),
 	FTRACE_DISABLE_CALLS		= (1 << 1),
 	FTRACE_UPDATE_TRACE_FUNC	= (1 << 2),
 	FTRACE_START_FUNC_RET		= (1 << 3),
@@ -1519,7 +1519,7 @@ int ftrace_text_reserved(void *start, void *end)
 
 
 static int
-__ftrace_replace_code(struct dyn_ftrace *rec, int enable)
+__ftrace_replace_code(struct dyn_ftrace *rec, int update)
 {
 	unsigned long ftrace_addr;
 	unsigned long flag = 0UL;
@@ -1527,17 +1527,17 @@ __ftrace_replace_code(struct dyn_ftrace *rec, int enable)
 	ftrace_addr = (unsigned long)FTRACE_ADDR;
 
 	/*
-	 * If we are enabling tracing:
+	 * If we are updating calls:
 	 *
 	 *   If the record has a ref count, then we need to enable it
 	 *   because someone is using it.
 	 *
 	 *   Otherwise we make sure its disabled.
 	 *
-	 * If we are disabling tracing, then disable all records that
+	 * If we are disabling calls, then disable all records that
 	 * are enabled.
 	 */
-	if (enable && (rec->flags & ~FTRACE_FL_MASK))
+	if (update && (rec->flags & ~FTRACE_FL_MASK))
 		flag = FTRACE_FL_ENABLED;
 
 	/* If the state of this record hasn't changed, then do nothing */
@@ -1553,7 +1553,7 @@ __ftrace_replace_code(struct dyn_ftrace *rec, int enable)
 	return ftrace_make_nop(NULL, rec, ftrace_addr);
 }
 
-static void ftrace_replace_code(int enable)
+static void ftrace_replace_code(int update)
 {
 	struct dyn_ftrace *rec;
 	struct ftrace_page *pg;
@@ -1567,7 +1567,7 @@ static void ftrace_replace_code(int enable)
 		if (rec->flags & FTRACE_FL_FREE)
 			continue;
 
-		failed = __ftrace_replace_code(rec, enable);
+		failed = __ftrace_replace_code(rec, update);
 		if (failed) {
 			ftrace_bug(failed, rec->ip);
 			/* Stop processing */
@@ -1623,7 +1623,7 @@ static int __ftrace_modify_code(void *data)
 	 */
 	function_trace_stop++;
 
-	if (*command & FTRACE_ENABLE_CALLS)
+	if (*command & FTRACE_UPDATE_CALLS)
 		ftrace_replace_code(1);
 	else if (*command & FTRACE_DISABLE_CALLS)
 		ftrace_replace_code(0);
@@ -1691,7 +1691,7 @@ static int ftrace_startup(struct ftrace_ops *ops, int command)
 		return -ENODEV;
 
 	ftrace_start_up++;
-	command |= FTRACE_ENABLE_CALLS;
+	command |= FTRACE_UPDATE_CALLS;
 
 	/* ops marked global share the filter hashes */
 	if (ops->flags & FTRACE_OPS_FL_GLOBAL) {
@@ -1743,8 +1743,7 @@ static void ftrace_shutdown(struct ftrace_ops *ops, int command)
 	if (ops != &global_ops || !global_start_up)
 		ops->flags &= ~FTRACE_OPS_FL_ENABLED;
 
-	if (!ftrace_start_up)
-		command |= FTRACE_DISABLE_CALLS;
+	command |= FTRACE_UPDATE_CALLS;
 
 	if (saved_ftrace_func != ftrace_trace_function) {
 		saved_ftrace_func = ftrace_trace_function;
@@ -1766,7 +1765,7 @@ static void ftrace_startup_sysctl(void)
 	saved_ftrace_func = NULL;
 	/* ftrace_start_up is true if we want ftrace running */
 	if (ftrace_start_up)
-		ftrace_run_update_code(FTRACE_ENABLE_CALLS);
+		ftrace_run_update_code(FTRACE_UPDATE_CALLS);
 }
 
 static void ftrace_shutdown_sysctl(void)
@@ -2919,7 +2918,7 @@ ftrace_set_regex(struct ftrace_ops *ops, unsigned char *buf, int len,
 	ret = ftrace_hash_move(ops, enable, orig_hash, hash);
 	if (!ret && ops->flags & FTRACE_OPS_FL_ENABLED
 	    && ftrace_enabled)
-		ftrace_run_update_code(FTRACE_ENABLE_CALLS);
+		ftrace_run_update_code(FTRACE_UPDATE_CALLS);
 
 	mutex_unlock(&ftrace_lock);
 
@@ -3107,7 +3106,7 @@ ftrace_regex_release(struct inode *inode, struct file *file)
 				       orig_hash, iter->hash);
 		if (!ret && (iter->ops->flags & FTRACE_OPS_FL_ENABLED)
 		    && ftrace_enabled)
-			ftrace_run_update_code(FTRACE_ENABLE_CALLS);
+			ftrace_run_update_code(FTRACE_UPDATE_CALLS);
 
 		mutex_unlock(&ftrace_lock);
 	}
author
#52 lore raw

PATCH v2 3/10 ftrace: Add enable/disable ftrace_ops control interface

In reply to Jiri Olsa

Adding a way to temporarily enable/disable ftrace_ops. The change follows the same way as 'global' ftrace_ops are done.

Introducing 2 global ftrace_ops - control_ops and ftrace_control_list which take over all ftrace_ops registered with FTRACE_OPS_FL_CONTROL flag. In addition new per cpu flag called 'disabled' is also added to ftrace_ops to provide the control information for each cpu.

When ftrace_ops with FTRACE_OPS_FL_CONTROL is registered, it is set as disabled for all cpus.

The ftrace_control_list contains all the registered 'control' ftrace_ops. The control_ops provides function which iterates ftrace_control_list and does the check for 'disabled' flag on current cpu.

Adding 2 inline functions enable_ftrace_function/disable_ftrace_function, which enable/disable the ftrace_ops for current cpu.

---
 include/linux/ftrace.h |   14 ++++++
 kernel/trace/ftrace.c  |  115 +++++++++++++++++++++++++++++++++++++++++++-----
 kernel/trace/trace.h   |    2 +
 3 files changed, 120 insertions(+), 11 deletions(-)
include/linux/ftrace.h +14 -0
index 26eafce..b223944 100644
@@ -35,12 +35,14 @@ enum {
 	FTRACE_OPS_FL_ENABLED		= 1 << 0,
 	FTRACE_OPS_FL_GLOBAL		= 1 << 1,
 	FTRACE_OPS_FL_DYNAMIC		= 1 << 2,
+	FTRACE_OPS_FL_CONTROL		= 1 << 3,
 };
 
 struct ftrace_ops {
 	ftrace_func_t			func;
 	struct ftrace_ops		*next;
 	unsigned long			flags;
+	void __percpu			*disabled;
 #ifdef CONFIG_DYNAMIC_FTRACE
 	struct ftrace_hash		*notrace_hash;
 	struct ftrace_hash		*filter_hash;
@@ -97,6 +99,18 @@ int register_ftrace_function(struct ftrace_ops *ops);
 int unregister_ftrace_function(struct ftrace_ops *ops);
 void clear_ftrace_function(void);
 
+static inline void enable_ftrace_function(struct ftrace_ops *ops)
+{
+	atomic_t *disabled = this_cpu_ptr(ops->disabled);
+	atomic_dec(disabled);
+}
+
+static inline void disable_ftrace_function(struct ftrace_ops *ops)
+{
+	atomic_t *disabled = this_cpu_ptr(ops->disabled);
+	atomic_inc(disabled);
+}
+
 extern void ftrace_stub(unsigned long a0, unsigned long a1);
 
 #else /* !CONFIG_FUNCTION_TRACER */
kernel/trace/ftrace.c +104 -11

178 lines not on this page · raw message

kernel/trace/trace.h +2 -0
index f8ec229..da05926 100644
@@ -288,6 +288,8 @@ struct tracer {
 /* for function tracing recursion */
 #define TRACE_INTERNAL_BIT		(1<<11)
 #define TRACE_GLOBAL_BIT		(1<<12)
+#define TRACE_CONTROL_BIT		(1<<13)
+
 /*
  * Abuse of the trace_recursion.
  * As we need a way to maintain state if we are tracing the function
Signature · 1 line
1.7.1
maintainer
#53 lore raw

Re: [PATCHv2 03/10] ftrace: Add enable/disable ftrace_ops control interface

Quoted 122 lines · Jiri Olsa

Adding a way to temporarily enable/disable ftrace_ops. The change follows the same way as 'global' ftrace_ops are done.

Introducing 2 global ftrace_ops - control_ops and ftrace_control_list which take over all ftrace_ops registered with FTRACE_OPS_FL_CONTROL flag. In addition new per cpu flag called 'disabled' is also added to ftrace_ops to provide the control information for each cpu.

When ftrace_ops with FTRACE_OPS_FL_CONTROL is registered, it is set as disabled for all cpus.

The ftrace_control_list contains all the registered 'control' ftrace_ops. The control_ops provides function which iterates ftrace_control_list and does the check for 'disabled' flag on current cpu.

Adding 2 inline functions enable_ftrace_function/disable_ftrace_function, which enable/disable the ftrace_ops for current cpu.

---
 include/linux/ftrace.h |   14 ++++++
 kernel/trace/ftrace.c  |  115 +++++++++++++++++++++++++++++++++++++++++++-----
 kernel/trace/trace.h   |    2 +
 3 files changed, 120 insertions(+), 11 deletions(-)
include/linux/ftrace.h +14 -0
index 26eafce..b223944 100644
@@ -35,12 +35,14 @@ enum {
 	FTRACE_OPS_FL_ENABLED		= 1 << 0,
 	FTRACE_OPS_FL_GLOBAL		= 1 << 1,
 	FTRACE_OPS_FL_DYNAMIC		= 1 << 2,
+	FTRACE_OPS_FL_CONTROL		= 1 << 3,
 };
 
 struct ftrace_ops {
 	ftrace_func_t			func;
 	struct ftrace_ops		*next;
 	unsigned long			flags;
+	void __percpu			*disabled;
 #ifdef CONFIG_DYNAMIC_FTRACE
 	struct ftrace_hash		*notrace_hash;
 	struct ftrace_hash		*filter_hash;
@@ -97,6 +99,18 @@ int register_ftrace_function(struct ftrace_ops *ops);
 int unregister_ftrace_function(struct ftrace_ops *ops);
 void clear_ftrace_function(void);
 
+static inline void enable_ftrace_function(struct ftrace_ops *ops)
+{
+	atomic_t *disabled = this_cpu_ptr(ops->disabled);
+	atomic_dec(disabled);
+}
+
+static inline void disable_ftrace_function(struct ftrace_ops *ops)
+{
+	atomic_t *disabled = this_cpu_ptr(ops->disabled);
+	atomic_inc(disabled);
+}
+
 extern void ftrace_stub(unsigned long a0, unsigned long a1);
 
 #else /* !CONFIG_FUNCTION_TRACER */
kernel/trace/ftrace.c +34 -0
index b79ab33..c2fa233 100644
@@ -87,12 +87,14 @@ static struct ftrace_ops ftrace_list_end __read_mostly = {
 };
 
 static struct ftrace_ops *ftrace_global_list __read_mostly = &ftrace_list_end;
+static struct ftrace_ops *ftrace_control_list __read_mostly = &ftrace_list_end;
 static struct ftrace_ops *ftrace_ops_list __read_mostly = &ftrace_list_end;
 ftrace_func_t ftrace_trace_function __read_mostly = ftrace_stub;
 static ftrace_func_t __ftrace_trace_function_delay __read_mostly = ftrace_stub;
 ftrace_func_t __ftrace_trace_function __read_mostly = ftrace_stub;
 ftrace_func_t ftrace_pid_function __read_mostly = ftrace_stub;
 static struct ftrace_ops global_ops;
+static struct ftrace_ops control_ops;
 
 static void
 ftrace_ops_list_func(unsigned long ip, unsigned long parent_ip);
@@ -166,6 +168,38 @@ static void ftrace_test_stop_func(unsigned long ip, unsigned long parent_ip)
 }
 #endif
 
+static void control_ops_disable_all(struct ftrace_ops *ops)
+{
+	int cpu;
+
+	for_each_possible_cpu(cpu)
+		atomic_set(per_cpu_ptr(ops->disabled, cpu), 1);
+}
+
+static int control_ops_alloc(struct ftrace_ops *ops)
+{
+	atomic_t *disabled;
+
+	disabled = alloc_percpu(atomic_t);
+	if (!disabled)
+		return -ENOMEM;
+
+	ops->disabled = disabled;
+	control_ops_disable_all(ops);
+	return 0;
+}
+
+static void control_ops_free(struct ftrace_ops *ops)
+{
+	free_percpu(ops->disabled);
+}
+
+static int control_ops_is_disabled(struct ftrace_ops *ops)
+{
+	atomic_t *disabled = this_cpu_ptr(ops->disabled);
+	return atomic_read(disabled);
+}
+
 static void update_global_ops(void)
 {
 	ftrace_func_t func;

kernel/trace/ftrace.c static void update_ftrace_function(void) · line 258

Quoted 5 lines
kernel/trace/ftrace.c +1 -1
@@ -221,7 +255,7 @@ static void update_ftrace_function(void)
 #endif
 }
 
-static void add_ftrace_ops(struct ftrace_ops **list, struct ftrace_ops *ops)
+static void __add_ftrace_ops(struct ftrace_ops **list, struct ftrace_ops *ops)

Don't do the above (see below).

Quoted 13 lines
kernel/trace/ftrace.c +1 -1
 {
 	ops->next = *list;
 	/*
@@ -233,7 +267,7 @@ static void add_ftrace_ops(struct ftrace_ops **list, struct ftrace_ops *ops)
 	rcu_assign_pointer(*list, ops);
 }
 
-static int remove_ftrace_ops(struct ftrace_ops **list, struct ftrace_ops *ops)
+static int __remove_ftrace_ops(struct ftrace_ops **list, struct ftrace_ops *ops)
 {
 	struct ftrace_ops **p;
 

kernel/trace/ftrace.c static int remove_ftrace_ops(struct ftrace_ops **list, struct ftrace_ops *ops) · line 296

Quoted 6 lines
kernel/trace/ftrace.c +3 -0
@@ -257,6 +291,26 @@ static int remove_ftrace_ops(struct ftrace_ops **list, struct ftrace_ops *ops)
 	return 0;
 }
 
+static void add_ftrace_ops(struct ftrace_ops **list,
+			   struct ftrace_ops *main_ops,
+			   struct ftrace_ops *ops)

Call this add_ftrace_list_ops or something (see below).

Quoted 21 lines
kernel/trace/ftrace.c +17 -0
+{
+	int first = *list == &ftrace_list_end;
+	__add_ftrace_ops(list, ops);
+	if (first)
+		__add_ftrace_ops(&ftrace_ops_list, main_ops);
+}
+
+static int remove_ftrace_ops(struct ftrace_ops **list,
+			      struct ftrace_ops *main_ops,
+			      struct ftrace_ops *ops)
+{
+	int ret = __remove_ftrace_ops(list, ops);
+	if (!ret && *list == &ftrace_list_end)
+		ret = __remove_ftrace_ops(&ftrace_ops_list, main_ops);
+	return ret;
+}
+
 static int __register_ftrace_function(struct ftrace_ops *ops)
 {
 	if (ftrace_disabled)

kernel/trace/ftrace.c static int __register_ftrace_function(struct ftrace_ops *ops) · line 328

Quoted 7 lines
kernel/trace/ftrace.c +4 -0
@@ -268,17 +322,23 @@ static int __register_ftrace_function(struct ftrace_ops *ops)
 	if (WARN_ON(ops->flags & FTRACE_OPS_FL_ENABLED))
 		return -EBUSY;
 
+#define FL_GLOBAL_CONTROL (FTRACE_OPS_FL_GLOBAL | FTRACE_OPS_FL_CONTROL)
+	if ((ops->flags & FL_GLOBAL_CONTROL) == FL_GLOBAL_CONTROL)
+		return -EINVAL;
+#undef FL_GLOBAL_CONTROL

The above is ugly. Just define a FL_GLOBAL_CONTROL near the top or something.

Quoted 8 lines
kernel/trace/ftrace.c +2 -2
+
 	if (!core_kernel_data((unsigned long)ops))
 		ops->flags |= FTRACE_OPS_FL_DYNAMIC;
 
 	if (ops->flags & FTRACE_OPS_FL_GLOBAL) {
-		int first = ftrace_global_list == &ftrace_list_end;
-		add_ftrace_ops(&ftrace_global_list, ops);
+		add_ftrace_ops(&ftrace_global_list, &global_ops, ops);

kernel/trace/ftrace.c static int __register_ftrace_function(struct ftrace_ops *ops) · line 341

Quoted 10 lines
kernel/trace/ftrace.c +5 -3
 		ops->flags |= FTRACE_OPS_FL_ENABLED;
-		if (first)
-			add_ftrace_ops(&ftrace_ops_list, &global_ops);
+	} else if (ops->flags & FTRACE_OPS_FL_CONTROL) {
+		if (control_ops_alloc(ops))
+			return -ENOMEM;
+		add_ftrace_ops(&ftrace_control_list, &control_ops, ops);
 	} else
-		add_ftrace_ops(&ftrace_ops_list, ops);
+		__add_ftrace_ops(&ftrace_ops_list, ops);

Don't use the name __add_ftrace_ops() and add_ftrace_ops() it's confusing and error prone. Keep the add_ftrace_ops() as the original, and make your new function "add_ftrace_list_ops()" or something.

Quoted 7 lines
kernel/trace/ftrace.c +0 -0
 	if (ftrace_enabled)
 		update_ftrace_function();
@@ -300,13 +360,16 @@ static int __unregister_ftrace_function(struct ftrace_ops *ops)
 		return -EINVAL;
 
 	if (ops->flags & FTRACE_OPS_FL_GLOBAL) {

kernel/trace/ftrace.c static int __unregister_ftrace_function(struct ftrace_ops *ops) · line 369

Quoted 10 lines
kernel/trace/ftrace.c +5 -3
-		ret = remove_ftrace_ops(&ftrace_global_list, ops);
-		if (!ret && ftrace_global_list == &ftrace_list_end)
-			ret = remove_ftrace_ops(&ftrace_ops_list, &global_ops);
+		ret = remove_ftrace_ops(&ftrace_global_list, &global_ops, ops);
 		if (!ret)
 			ops->flags &= ~FTRACE_OPS_FL_ENABLED;
+	} else if (ops->flags & FTRACE_OPS_FL_CONTROL) {
+		ret = remove_ftrace_ops(&ftrace_control_list,
+					&control_ops, ops);
+		if (!ret)

You need a synchronize_sched() here, otherwise you can have functions accessing the ops->per_cpu variables. A process on another CPU could be about to use the ops, and although you removed it from the list, it is still being used on another CPU. If you free now without a synchronize_sched(), the other CPU can be using the freed variable.

kernel/trace/ftrace.c static int __unregister_ftrace_function(struct ftrace_ops *ops) · line 372

Quoted 4 lines
kernel/trace/ftrace.c +2 -1
+			control_ops_free(ops);
 	} else
-		ret = remove_ftrace_ops(&ftrace_ops_list, ops);
+		ret = __remove_ftrace_ops(&ftrace_ops_list, ops);

Same thing goes with the remove_ftrace_ops and __remove_ftrace_ops().

Quoted 20 lines
kernel/trace/ftrace.c +13 -0
 	if (ret < 0)
 		return ret;
@@ -3562,6 +3625,36 @@ ftrace_ops_test(struct ftrace_ops *ops, unsigned long ip)
 #endif /* CONFIG_DYNAMIC_FTRACE */
 
 static void
+ftrace_ops_control_func(unsigned long ip, unsigned long parent_ip)
+{
+	struct ftrace_ops *op;
+
+	if (unlikely(trace_recursion_test(TRACE_CONTROL_BIT)))
+		return;
+
+	/*
+	 * Some of the ops may be dynamically allocated,
+	 * they must be freed after a synchronize_sched().
+	 */
+	preempt_disable_notrace();
+	trace_recursion_set(TRACE_CONTROL_BIT);

kernel/trace/ftrace.c ftrace_ops_test(struct ftrace_ops *ops, unsigned long ip) · line 3650

Quoted 10 lines
kernel/trace/ftrace.c +10 -0
+	op = rcu_dereference_raw(ftrace_control_list);
+	while (op != &ftrace_list_end) {
+		if (!control_ops_is_disabled(op) &&
+		    ftrace_ops_test(op, ip))
+			op->func(ip, parent_ip);
+
+		op = rcu_dereference_raw(op->next);
+	};
+	preempt_enable_notrace();
+	trace_recursion_clear(TRACE_CONTROL_BIT);

Shouldn't the above be reversed to match the preempt_disable() and trace_recursion_set(). That is:

	preempt_disable();
	trace_recursion_set();
	[...]
	trace_recursion_clear();
	preempt_enable();

That would make in symmetric.

-- Steve

Quoted 23 lines
kernel/trace/ftrace.c +7 -0
+}
+
+static struct ftrace_ops control_ops = {
+	.func = ftrace_ops_control_func,
+};
+
+static void
 ftrace_ops_list_func(unsigned long ip, unsigned long parent_ip)
 {
 	struct ftrace_ops *op;
kernel/trace/trace.h +2 -0
index f8ec229..da05926 100644
@@ -288,6 +288,8 @@ struct tracer {
 /* for function tracing recursion */
 #define TRACE_INTERNAL_BIT		(1<<11)
 #define TRACE_GLOBAL_BIT		(1<<12)
+#define TRACE_CONTROL_BIT		(1<<13)
+
 /*
  * Abuse of the trace_recursion.
  * As we need a way to maintain state if we are tracing the function
maintainer
#54 lore raw

Re: [PATCHv2 03/10] ftrace: Add enable/disable ftrace_ops control interface

In reply to Jiri Olsa

include/linux/ftrace.h int register_ftrace_function(struct ftrace_ops *ops); · line 113

Quoted 13 lines · Jiri Olsa
include/linux/ftrace.h +12 -0
+static inline void enable_ftrace_function(struct ftrace_ops *ops)
+{
+	atomic_t *disabled = this_cpu_ptr(ops->disabled);
+	atomic_dec(disabled);
+}
+
+static inline void disable_ftrace_function(struct ftrace_ops *ops)
+{
+	atomic_t *disabled = this_cpu_ptr(ops->disabled);
+	atomic_inc(disabled);
+}
+

The above should be renamed to ftrace_function_enable/disable(), and they should pass in the cpu. There may be a case we want to disable ftrace functions on another CPU.

Not to mention, this is the perfect example of "this_cpu_ptr" being used incorrectly. It's not made for this purpose, and again the naming of "this_cpu" totally confuses other kernel developers. We need to change this name as it was agreed to at Kernel Summit.

If the above is called with preemption enabled, it will not do what is expected. We could disable function tracing on one CPU and then re-enable it for another CPU even though it is already enabled.

-- Steve

author
#55 lore raw

Re: [PATCHv2 03/10] ftrace: Add enable/disable ftrace_ops control interface

Quoted 19 lines · Steven Rostedt

include/linux/ftrace.h int register_ftrace_function(struct ftrace_ops *ops); · line 113

Quoted 13 lines · Jiri Olsa
include/linux/ftrace.h +12 -0
+static inline void enable_ftrace_function(struct ftrace_ops *ops)
+{
+	atomic_t *disabled = this_cpu_ptr(ops->disabled);
+	atomic_dec(disabled);
+}
+
+static inline void disable_ftrace_function(struct ftrace_ops *ops)
+{
+	atomic_t *disabled = this_cpu_ptr(ops->disabled);
+	atomic_inc(disabled);
+}
+

The above should be renamed to ftrace_function_enable/disable(), and they should pass in the cpu. There may be a case we want to disable ftrace functions on another CPU.

Quoted 4 lines

Not to mention, this is the perfect example of "this_cpu_ptr" being used incorrectly. It's not made for this purpose, and again the naming of "this_cpu" totally confuses other kernel developers. We need to change this name as it was agreed to at Kernel Summit.

ok, will make the changes

Quoted 4 lines

If the above is called with preemption enabled, it will not do what is expected. We could disable function tracing on one CPU and then re-enable it for another CPU even though it is already enabled.

It is only called inside perf reg callback within the schedule function where the preemption is disabled.

The ftrace_function_enable is called when task is scheduled in on respective cpu. Likewise the ftrace_function_disable is called when task is scheduled out on respective cpu.

thanks,
jirka

maintainer
#56 lore raw

Re: [PATCHv2 03/10] ftrace: Add enable/disable ftrace_ops control interface

Quoted 11 lines · Jiri Olsa
Quoted 4 lines

If the above is called with preemption enabled, it will not do what is expected. We could disable function tracing on one CPU and then re-enable it for another CPU even though it is already enabled.

It is only called inside perf reg callback within the schedule function where the preemption is disabled.

The ftrace_function_enable is called when task is scheduled in on respective cpu. Likewise the ftrace_function_disable is called when task is scheduled out on respective cpu.

Yes I know how you use it, but this is an open API. It may be currently only used by perf today, but that doesn't mean that it wont be used by others. There's no documentation on how to use it. I don't look at this and say, "oh this is used by perf, we only need to worry about how perf uses it". That doesn't scale. It needs to be documented on how to use it, and if it requires preemption disabled when calling it, it should definitely be stated that fact and it may need a WARN_ON(preempt_count()) or something.

-- Steve

author
#57 lore raw

Re: [PATCHv2 03/10] ftrace: Add enable/disable ftrace_ops control interface

Quoted 14 lines · Steven Rostedt
Quoted 11 lines · Jiri Olsa
Quoted 4 lines

If the above is called with preemption enabled, it will not do what is expected. We could disable function tracing on one CPU and then re-enable it for another CPU even though it is already enabled.

It is only called inside perf reg callback within the schedule function where the preemption is disabled.

The ftrace_function_enable is called when task is scheduled in on respective cpu. Likewise the ftrace_function_disable is called when task is scheduled out on respective cpu.

Quoted 8 lines

Yes I know how you use it, but this is an open API. It may be currently only used by perf today, but that doesn't mean that it wont be used by others. There's no documentation on how to use it. I don't look at this and say, "oh this is used by perf, we only need to worry about how perf uses it". That doesn't scale. It needs to be documented on how to use it, and if it requires preemption disabled when calling it, it should definitely be stated that fact and it may need a WARN_ON(preempt_count()) or something.

I've already added

        if (WARN_ON_ONCE(!(ops->flags & FTRACE_OPS_FL_CONTROL)))
                return;

will add similar for preemption and make comments for both functions

thanks,
jirka

Quoted 4 lines

-- Steve

author
#58 lore raw

PATCH v2 4/10 ftrace, perf: Add open/close tracepoint perf registration actions

In reply to Jiri Olsa

Adding TRACE_REG_PERF_OPEN and TRACE_REG_PERF_CLOSE to differentiate register/unregister from open/close actions.

The register/unregister actions are invoked for the first/last tracepoint user when opening/closing the evet.

The open/close actions are invoked for each tracepoint user when opening/closing the event.

---
 include/linux/ftrace_event.h    |    6 +-
 kernel/trace/trace.h            |    5 ++
 kernel/trace/trace_event_perf.c |  116 +++++++++++++++++++++++++--------------
 kernel/trace/trace_events.c     |   10 ++-
 kernel/trace/trace_kprobe.c     |    6 ++-
 kernel/trace/trace_syscalls.c   |   14 +++-
 6 files changed, 106 insertions(+), 51 deletions(-)
include/linux/ftrace_event.h +4 -2
index c3da42d..195e360 100644
@@ -146,6 +146,8 @@ enum trace_reg {
 	TRACE_REG_UNREGISTER,
 	TRACE_REG_PERF_REGISTER,
 	TRACE_REG_PERF_UNREGISTER,
+	TRACE_REG_PERF_OPEN,
+	TRACE_REG_PERF_CLOSE,
 };
 
 struct ftrace_event_call;
@@ -157,7 +159,7 @@ struct ftrace_event_class {
 	void			*perf_probe;
 #endif
 	int			(*reg)(struct ftrace_event_call *event,
-				       enum trace_reg type);
+				       enum trace_reg type, void *data);
 	int			(*define_fields)(struct ftrace_event_call *);
 	struct list_head	*(*get_fields)(struct ftrace_event_call *);
 	struct list_head	fields;
@@ -165,7 +167,7 @@ struct ftrace_event_class {
 };
 
 extern int ftrace_event_reg(struct ftrace_event_call *event,
-			    enum trace_reg type);
+			    enum trace_reg type, void *data);
 
 enum {
 	TRACE_EVENT_FL_ENABLED_BIT,
kernel/trace/trace.h +5 -0
index da05926..476527a 100644
@@ -827,4 +827,9 @@ extern const char *__stop___trace_bprintk_fmt[];
 	FTRACE_ENTRY(call, struct_name, id, PARAMS(tstruct), PARAMS(print))
 #include "trace_entries.h"
 
+#ifdef CONFIG_PERF_EVENTS
+int perf_ftrace_event_register(struct ftrace_event_call *call,
+			       enum trace_reg type, void *data);
+#endif
+
 #endif /* _LINUX_KERNEL_TRACE_H */
kernel/trace/trace_event_perf.c +75 -41

154 lines not on this page · raw message

kernel/trace/trace_events.c +7 -3
index c212a7f..5138fea 100644
@@ -147,7 +147,8 @@ int trace_event_raw_init(struct ftrace_event_call *call)
 }
 EXPORT_SYMBOL_GPL(trace_event_raw_init);
 
-int ftrace_event_reg(struct ftrace_event_call *call, enum trace_reg type)
+int ftrace_event_reg(struct ftrace_event_call *call,
+		     enum trace_reg type, void *data)
 {
 	switch (type) {
 	case TRACE_REG_REGISTER:
@@ -170,6 +171,9 @@ int ftrace_event_reg(struct ftrace_event_call *call, enum trace_reg type)
 					    call->class->perf_probe,
 					    call);
 		return 0;
+	case TRACE_REG_PERF_OPEN:
+	case TRACE_REG_PERF_CLOSE:
+		return 0;
 #endif
 	}
 	return 0;
@@ -209,7 +213,7 @@ static int ftrace_event_enable_disable(struct ftrace_event_call *call,
 				tracing_stop_cmdline_record();
 				call->flags &= ~TRACE_EVENT_FL_RECORDED_CMD;
 			}
-			call->class->reg(call, TRACE_REG_UNREGISTER);
+			call->class->reg(call, TRACE_REG_UNREGISTER, NULL);
 		}
 		break;
 	case 1:
@@ -218,7 +222,7 @@ static int ftrace_event_enable_disable(struct ftrace_event_call *call,
 				tracing_start_cmdline_record();
 				call->flags |= TRACE_EVENT_FL_RECORDED_CMD;
 			}
-			ret = call->class->reg(call, TRACE_REG_REGISTER);
+			ret = call->class->reg(call, TRACE_REG_REGISTER, NULL);
 			if (ret) {
 				tracing_stop_cmdline_record();
 				pr_info("event trace: Could not enable event "
kernel/trace/trace_kprobe.c +5 -1
index 00d527c..5667f89 100644
@@ -1892,7 +1892,8 @@ static __kprobes void kretprobe_perf_func(struct kretprobe_instance *ri,
 #endif	/* CONFIG_PERF_EVENTS */
 
 static __kprobes
-int kprobe_register(struct ftrace_event_call *event, enum trace_reg type)
+int kprobe_register(struct ftrace_event_call *event,
+		    enum trace_reg type, void *data)
 {
 	struct trace_probe *tp = (struct trace_probe *)event->data;
 
@@ -1909,6 +1910,9 @@ int kprobe_register(struct ftrace_event_call *event, enum trace_reg type)
 	case TRACE_REG_PERF_UNREGISTER:
 		disable_trace_probe(tp, TP_FLAG_PROFILE);
 		return 0;
+	case TRACE_REG_PERF_OPEN:
+	case TRACE_REG_PERF_CLOSE:
+		return 0;
 #endif
 	}
 	return 0;
kernel/trace/trace_syscalls.c +10 -4
index cb65454..6916b0d 100644
@@ -17,9 +17,9 @@ static DECLARE_BITMAP(enabled_enter_syscalls, NR_syscalls);
 static DECLARE_BITMAP(enabled_exit_syscalls, NR_syscalls);
 
 static int syscall_enter_register(struct ftrace_event_call *event,
-				 enum trace_reg type);
+				 enum trace_reg type, void *data);
 static int syscall_exit_register(struct ftrace_event_call *event,
-				 enum trace_reg type);
+				 enum trace_reg type, void *data);
 
 static int syscall_enter_define_fields(struct ftrace_event_call *call);
 static int syscall_exit_define_fields(struct ftrace_event_call *call);
@@ -649,7 +649,7 @@ void perf_sysexit_disable(struct ftrace_event_call *call)
 #endif /* CONFIG_PERF_EVENTS */
 
 static int syscall_enter_register(struct ftrace_event_call *event,
-				 enum trace_reg type)
+				 enum trace_reg type, void *data)
 {
 	switch (type) {
 	case TRACE_REG_REGISTER:
@@ -664,13 +664,16 @@ static int syscall_enter_register(struct ftrace_event_call *event,
 	case TRACE_REG_PERF_UNREGISTER:
 		perf_sysenter_disable(event);
 		return 0;
+	case TRACE_REG_PERF_OPEN:
+	case TRACE_REG_PERF_CLOSE:
+		return 0;
 #endif
 	}
 	return 0;
 }
 
 static int syscall_exit_register(struct ftrace_event_call *event,
-				 enum trace_reg type)
+				 enum trace_reg type, void *data)
 {
 	switch (type) {
 	case TRACE_REG_REGISTER:
@@ -685,6 +688,9 @@ static int syscall_exit_register(struct ftrace_event_call *event,
 	case TRACE_REG_PERF_UNREGISTER:
 		perf_sysexit_disable(event);
 		return 0;
+	case TRACE_REG_PERF_OPEN:
+	case TRACE_REG_PERF_CLOSE:
+		return 0;
 #endif
 	}
 	return 0;
Signature · 1 line
1.7.1
author
#59 lore raw

PATCH v2 5/10 ftrace, perf: Add add/del tracepoint perf registration actions

In reply to Jiri Olsa

Adding TRACE_REG_PERF_ADD and TRACE_REG_PERF_DEL to handle perf event schedule in/out actions.

The add action is invoked for when the perf event is scheduled in, while the del action is invoked when the event is scheduled out.

---
 include/linux/ftrace_event.h    |    2 ++
 kernel/trace/trace_event_perf.c |    4 +++-
 kernel/trace/trace_events.c     |    2 ++
 kernel/trace/trace_kprobe.c     |    2 ++
 kernel/trace/trace_syscalls.c   |    4 ++++
 5 files changed, 13 insertions(+), 1 deletions(-)
include/linux/ftrace_event.h +2 -0
index 195e360..2bf677c 100644
@@ -148,6 +148,8 @@ enum trace_reg {
 	TRACE_REG_PERF_UNREGISTER,
 	TRACE_REG_PERF_OPEN,
 	TRACE_REG_PERF_CLOSE,
+	TRACE_REG_PERF_ADD,
+	TRACE_REG_PERF_DEL,
 };
 
 struct ftrace_event_call;
kernel/trace/trace_event_perf.c +3 -1
index 0cfcc37..d72af0b 100644
@@ -211,12 +211,14 @@ int perf_trace_add(struct perf_event *p_event, int flags)
 	list = this_cpu_ptr(pcpu_list);
 	hlist_add_head_rcu(&p_event->hlist_entry, list);
 
-	return 0;
+	return tp_event->class->reg(tp_event, TRACE_REG_PERF_ADD, p_event);
 }
 
 void perf_trace_del(struct perf_event *p_event, int flags)
 {
+	struct ftrace_event_call *tp_event = p_event->tp_event;
 	hlist_del_rcu(&p_event->hlist_entry);
+	tp_event->class->reg(tp_event, TRACE_REG_PERF_DEL, p_event);
 }
 
 __kprobes void *perf_trace_buf_prepare(int size, unsigned short type,
kernel/trace/trace_events.c +2 -0
index 5138fea..079a93a 100644
@@ -173,6 +173,8 @@ int ftrace_event_reg(struct ftrace_event_call *call,
 		return 0;
 	case TRACE_REG_PERF_OPEN:
 	case TRACE_REG_PERF_CLOSE:
+	case TRACE_REG_PERF_ADD:
+	case TRACE_REG_PERF_DEL:
 		return 0;
 #endif
 	}
kernel/trace/trace_kprobe.c +2 -0
index 5667f89..580a05e 100644
@@ -1912,6 +1912,8 @@ int kprobe_register(struct ftrace_event_call *event,
 		return 0;
 	case TRACE_REG_PERF_OPEN:
 	case TRACE_REG_PERF_CLOSE:
+	case TRACE_REG_PERF_ADD:
+	case TRACE_REG_PERF_DEL:
 		return 0;
 #endif
 	}
kernel/trace/trace_syscalls.c +4 -0
index 6916b0d..dbdd804 100644
@@ -666,6 +666,8 @@ static int syscall_enter_register(struct ftrace_event_call *event,
 		return 0;
 	case TRACE_REG_PERF_OPEN:
 	case TRACE_REG_PERF_CLOSE:
+	case TRACE_REG_PERF_ADD:
+	case TRACE_REG_PERF_DEL:
 		return 0;
 #endif
 	}
@@ -690,6 +692,8 @@ static int syscall_exit_register(struct ftrace_event_call *event,
 		return 0;
 	case TRACE_REG_PERF_OPEN:
 	case TRACE_REG_PERF_CLOSE:
+	case TRACE_REG_PERF_ADD:
+	case TRACE_REG_PERF_DEL:
 		return 0;
 #endif
 	}
Signature · 1 line
1.7.1
author
#60 lore raw

PATCH v2 6/10 ftrace, perf: Add support to use function tracepoint in perf

In reply to Jiri Olsa

Adding perf registration support for the ftrace function event, so it is now possible to register it via perf interface.

The perf_event struct statically contains ftrace_ops as a handle for function tracer. The function tracer is registered/unregistered in open/close actions, and enabled/disabled in add/del actions.

It is now possible to use function trace within perf commands like:

perf record -e ftrace:function ls
perf stat -e ftrace:function ls

Allowed only for root.

---
 include/linux/perf_event.h      |    3 +
 kernel/trace/trace.h            |    2 +
 kernel/trace/trace_event_perf.c |   92 +++++++++++++++++++++++++++++++++++++++
 kernel/trace/trace_export.c     |   28 ++++++++++++
 4 files changed, 125 insertions(+), 0 deletions(-)
include/linux/perf_event.h +3 -0
index 1e9ebe5..6071995 100644
@@ -847,6 +847,9 @@ struct perf_event {
 #ifdef CONFIG_EVENT_TRACING
 	struct ftrace_event_call	*tp_event;
 	struct event_filter		*filter;
+#ifdef CONFIG_FUNCTION_TRACER
+	struct ftrace_ops               ftrace_ops;
+#endif
 #endif
 
 #ifdef CONFIG_CGROUP_PERF
kernel/trace/trace.h +2 -0
index 476527a..4199916 100644
@@ -591,6 +591,8 @@ static inline int ftrace_trace_task(struct task_struct *task)
 static inline int ftrace_is_dead(void) { return 0; }
 #endif
 
+int ftrace_event_is_function(struct ftrace_event_call *call);
+
 /*
  * struct trace_parser - servers for reading the user input separated by spaces
  * @cont: set if the input is not complete - no final space char was found
kernel/trace/trace_event_perf.c +92 -0

101 lines not on this page · raw message

kernel/trace/trace_export.c +28 -0
index bbeec31..867653c 100644
@@ -131,6 +131,28 @@ ftrace_define_fields_##name(struct ftrace_event_call *event_call)	\
 
 #include "trace_entries.h"
 
+static int ftrace_event_class_register(struct ftrace_event_call *call,
+				       enum trace_reg type, void *data)
+{
+	switch (type) {
+	case TRACE_REG_PERF_REGISTER:
+	case TRACE_REG_PERF_UNREGISTER:
+		return 0;
+	case TRACE_REG_PERF_OPEN:
+	case TRACE_REG_PERF_CLOSE:
+	case TRACE_REG_PERF_ADD:
+	case TRACE_REG_PERF_DEL:
+#ifdef CONFIG_PERF_EVENTS
+		return perf_ftrace_event_register(call, type, data);
+#endif
+	case TRACE_REG_REGISTER:
+	case TRACE_REG_UNREGISTER:
+		break;
+	}
+
+	return -EINVAL;
+}
+
 #undef __entry
 #define __entry REC
 
@@ -159,6 +181,7 @@ struct ftrace_event_class event_class_ftrace_##call = {			\
 	.system			= __stringify(TRACE_SYSTEM),		\
 	.define_fields		= ftrace_define_fields_##call,		\
 	.fields			= LIST_HEAD_INIT(event_class_ftrace_##call.fields),\
+	.reg			= ftrace_event_class_register,		\
 };									\
 									\
 struct ftrace_event_call __used event_##call = {			\
@@ -170,4 +193,9 @@ struct ftrace_event_call __used event_##call = {			\
 struct ftrace_event_call __used						\
 __attribute__((section("_ftrace_events"))) *__event_##call = &event_##call;
 
+int ftrace_event_is_function(struct ftrace_event_call *call)
+{
+	return call == &event_function;
+}
+
 #include "trace_entries.h"
Signature · 1 line
1.7.1
author
#61 lore raw

PATCH v2 7/10 ftrace: Change filter/notrace set functions to return exit code

In reply to Jiri Olsa

Currently the ftrace_set_filter and ftrace_set_notrace functions do not return any return code. So there's no way for ftrace_ops user to tell wether the filter was correctly applied.

The set_ftrace_filter interface returns error in case the filter did not match:

  # echo krava > set_ftrace_filter
  bash: echo: write error: Invalid argument

Changing both ftrace_set_filter and ftrace_set_notrace functions to return zero if the filter was applied correctly or -E* values in case of error.

---
 include/linux/ftrace.h |    4 ++--
 kernel/trace/ftrace.c  |   16 ++++++++++------
 2 files changed, 12 insertions(+), 8 deletions(-)
include/linux/ftrace.h +2 -2
index b223944..ddd55cc 100644
@@ -194,9 +194,9 @@ struct dyn_ftrace {
 };
 
 int ftrace_force_update(void);
-void ftrace_set_filter(struct ftrace_ops *ops, unsigned char *buf,
+int ftrace_set_filter(struct ftrace_ops *ops, unsigned char *buf,
 		       int len, int reset);
-void ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
+int ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
 			int len, int reset);
 void ftrace_set_global_filter(unsigned char *buf, int len, int reset);
 void ftrace_set_global_notrace(unsigned char *buf, int len, int reset);
kernel/trace/ftrace.c +10 -6
index c2fa233..2dae0c7 100644
@@ -2975,8 +2975,11 @@ ftrace_set_regex(struct ftrace_ops *ops, unsigned char *buf, int len,
 	mutex_lock(&ftrace_regex_lock);
 	if (reset)
 		ftrace_filter_reset(hash);
-	if (buf)
-		ftrace_match_records(hash, buf, len);
+	if (buf &&
+	    !ftrace_match_records(hash, buf, len)) {
+		ret = -EINVAL;
+		goto out_regex_unlock;
+	}
 
 	mutex_lock(&ftrace_lock);
 	ret = ftrace_hash_move(ops, enable, orig_hash, hash);
@@ -2986,6 +2989,7 @@ ftrace_set_regex(struct ftrace_ops *ops, unsigned char *buf, int len,
 
 	mutex_unlock(&ftrace_lock);
 
+ out_regex_unlock:
 	mutex_unlock(&ftrace_regex_lock);
 
 	free_ftrace_hash(hash);
@@ -3002,10 +3006,10 @@ ftrace_set_regex(struct ftrace_ops *ops, unsigned char *buf, int len,
  * Filters denote which functions should be enabled when tracing is enabled.
  * If @buf is NULL and reset is set, all functions will be enabled for tracing.
  */
-void ftrace_set_filter(struct ftrace_ops *ops, unsigned char *buf,
+int ftrace_set_filter(struct ftrace_ops *ops, unsigned char *buf,
 		       int len, int reset)
 {
-	ftrace_set_regex(ops, buf, len, reset, 1);
+	return ftrace_set_regex(ops, buf, len, reset, 1);
 }
 EXPORT_SYMBOL_GPL(ftrace_set_filter);
 
@@ -3020,10 +3024,10 @@ EXPORT_SYMBOL_GPL(ftrace_set_filter);
  * is enabled. If @buf is NULL and reset is set, all functions will be enabled
  * for tracing.
  */
-void ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
+int ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
 			int len, int reset)
 {
-	ftrace_set_regex(ops, buf, len, reset, 0);
+	return ftrace_set_regex(ops, buf, len, reset, 0);
 }
 EXPORT_SYMBOL_GPL(ftrace_set_notrace);
 /**
Signature · 1 line
1.7.1
maintainer
#62 lore raw

Re: [PATCHv2 07/10] ftrace: Change filter/notrace set functions to return exit code

Quoted 7 lines · Jiri Olsa

Currently the ftrace_set_filter and ftrace_set_notrace functions do not return any return code. So there's no way for ftrace_ops user to tell wether the filter was correctly applied.

The set_ftrace_filter interface returns error in case the filter did not match:

Quoted 6 lines
  # echo krava > set_ftrace_filter
  bash: echo: write error: Invalid argument

Changing both ftrace_set_filter and ftrace_set_notrace functions to return zero if the filter was applied correctly or -E* values in case of error.

This looks like a proper fix. Move this to the front of the patch set, as this and patch 1 can go into ftrace now as fixes.

-- Steve

Quoted 2 lines
author
#63 lore raw

PATCH v2 8/10 ftrace, perf: Distinguish ftrace function event field type

In reply to Jiri Olsa

Adding FILTER_TRACE_FN event field type for function tracepoint event, so it can be properly recognized within filtering code.

Currently all fields of ftrace subsystem events share the common field type FILTER_OTHER. Since the function trace fields need special care within the filtering code we need to recognize it properly, hence adding the FILTER_TRACE_FN event type.

---
 include/linux/ftrace_event.h       |    1 +
 kernel/trace/trace_events_filter.c |    7 ++++++-
 kernel/trace/trace_export.c        |   25 ++++++++++++++++++++-----
 3 files changed, 27 insertions(+), 6 deletions(-)
include/linux/ftrace_event.h +1 -0
index 2bf677c..dd478fc 100644
@@ -245,6 +245,7 @@ enum {
 	FILTER_STATIC_STRING,
 	FILTER_DYN_STRING,
 	FILTER_PTR_STRING,
+	FILTER_TRACE_FN,
 };
 
 #define EVENT_STORAGE_SIZE 128
kernel/trace/trace_events_filter.c +6 -1
index fdc6d22..7b0b04c 100644
@@ -900,6 +900,11 @@ int filter_assign_type(const char *type)
 	return FILTER_OTHER;
 }
 
+static bool is_function_field(struct ftrace_event_field *field)
+{
+	return field->filter_type == FILTER_TRACE_FN;
+}
+
 static bool is_string_field(struct ftrace_event_field *field)
 {
 	return field->filter_type == FILTER_DYN_STRING ||
@@ -987,7 +992,7 @@ static int init_pred(struct filter_parse_state *ps,
 			fn = filter_pred_strloc;
 		else
 			fn = filter_pred_pchar;
-	} else {
+	} else if (!is_function_field(field)) {
 		if (field->is_signed)
 			ret = strict_strtoll(pred->regex.pattern, 0, &val);
 		else
kernel/trace/trace_export.c +20 -5
index 867653c..46c35e2 100644
@@ -67,7 +67,7 @@ static void __always_unused ____ftrace_check_##name(void)	\
 	ret = trace_define_field(event_call, #type, #item,		\
 				 offsetof(typeof(field), item),		\
 				 sizeof(field.item),			\
-				 is_signed_type(type), FILTER_OTHER);	\
+				 is_signed_type(type), filter_type);	\
 	if (ret)							\
 		return ret;
 
@@ -77,7 +77,7 @@ static void __always_unused ____ftrace_check_##name(void)	\
 				 offsetof(typeof(field),		\
 					  container.item),		\
 				 sizeof(field.container.item),		\
-				 is_signed_type(type), FILTER_OTHER);	\
+				 is_signed_type(type), filter_type);	\
 	if (ret)							\
 		return ret;
 
@@ -91,7 +91,7 @@ static void __always_unused ____ftrace_check_##name(void)	\
 		ret = trace_define_field(event_call, event_storage, #item, \
 				 offsetof(typeof(field), item),		\
 				 sizeof(field.item),			\
-				 is_signed_type(type), FILTER_OTHER);	\
+				 is_signed_type(type), filter_type);	\
 		mutex_unlock(&event_storage_mutex);			\
 		if (ret)						\
 			return ret;					\
@@ -104,7 +104,7 @@ static void __always_unused ____ftrace_check_##name(void)	\
 				 offsetof(typeof(field),		\
 					  container.item),		\
 				 sizeof(field.container.item),		\
-				 is_signed_type(type), FILTER_OTHER);	\
+				 is_signed_type(type), filter_type);	\
 	if (ret)							\
 		return ret;
 
@@ -112,10 +112,24 @@ static void __always_unused ____ftrace_check_##name(void)	\
 #define __dynamic_array(type, item)					\
 	ret = trace_define_field(event_call, #type, #item,		\
 				 offsetof(typeof(field), item),		\
-				 0, is_signed_type(type), FILTER_OTHER);\
+				 0, is_signed_type(type), filter_type);\
 	if (ret)							\
 		return ret;
 
+#define FILTER_TYPE_TRACE_FN           FILTER_TRACE_FN
+#define FILTER_TYPE_TRACE_GRAPH_ENT    FILTER_OTHER
+#define FILTER_TYPE_TRACE_GRAPH_RET    FILTER_OTHER
+#define FILTER_TYPE_TRACE_CTX          FILTER_OTHER
+#define FILTER_TYPE_TRACE_WAKE         FILTER_OTHER
+#define FILTER_TYPE_TRACE_STACK                FILTER_OTHER
+#define FILTER_TYPE_TRACE_USER_STACK   FILTER_OTHER
+#define FILTER_TYPE_TRACE_BPRINT       FILTER_OTHER
+#define FILTER_TYPE_TRACE_PRINT                FILTER_OTHER
+#define FILTER_TYPE_TRACE_MMIO_RW      FILTER_OTHER
+#define FILTER_TYPE_TRACE_MMIO_MAP     FILTER_OTHER
+#define FILTER_TYPE_TRACE_BRANCH       FILTER_OTHER
+#define FILTER_TYPE(arg)               FILTER_TYPE_##arg
+
 #undef FTRACE_ENTRY
 #define FTRACE_ENTRY(name, struct_name, id, tstruct, print)		\
 int									\
@@ -123,6 +137,7 @@ ftrace_define_fields_##name(struct ftrace_event_call *event_call)	\
 {									\
 	struct struct_name field;					\
 	int ret;							\
+	int filter_type = FILTER_TYPE(id);				\
 									\
 	tstruct;							\
 									\
Signature · 1 line
1.7.1
author
#64 lore raw

PATCH v2 9/10 ftrace, perf: Add filter support for function trace event

In reply to Jiri Olsa

Adding support to filter function trace event via perf interface. It is now possible to use filter interface in the perf tool like:

  perf record -e ftrace:function --filter="(ip == mm_*)" ls

The filter syntax is restricted to the the 'ip' field only, and following operators are accepted '==' '!=' '||', ending up with the filter strings like:

"ip == f1 f2 ..." || "ip != f3 f4 ..." ...

The '==' operator adds trace filter with same effect as would be added via set_ftrace_filter file.

The '!=' operator adds trace filter with same effect as would be added via set_ftrace_notrace file.

The right side of the '!=', '==' operators is list of functions or regexp. to be added to filter separated by space. Same syntax is supported/required as for the set_ftrace_filter and set_ftrace_notrace files.

The '||' operator is used for connecting multiple filter definitions together. It is possible to have more than one '==' and '!=' operators within one filter string.

---
 kernel/trace/trace.h               |    2 -
 kernel/trace/trace_events_filter.c |  113 +++++++++++++++++++++++++++++++++---
 2 files changed, 105 insertions(+), 10 deletions(-)
kernel/trace/trace.h +0 -2
index 4199916..9b7a004 100644
@@ -769,9 +769,7 @@ struct filter_pred {
 	u64 			val;
 	struct regex		regex;
 	unsigned short		*ops;
-#ifdef CONFIG_FTRACE_STARTUP_TEST
 	struct ftrace_event_field *field;
-#endif
 	int 			offset;
 	int 			not;
 	int 			op;
kernel/trace/trace_events_filter.c +105 -8

173 lines not on this page · raw message

Signature · 1 line
1.7.1
author
#65 lore raw

PATCH v2 10/10 ftrace, graph: Add global_ops filter callback for graph tracing

In reply to Jiri Olsa

The function graph tracer should depend on the global_ops filter, and process only functions that pass the global_ops filter.

Currently the function graph tracer gets all the functions enabled for tracing no matter what ftrace_ops enabled them.

Adding a hook for the graph entry callback, which ensures the function is compared against the global_ops filter and bail out of if it does not match.

This hook is enabled only if there's at least one non global ftrace_ops registered.

---
 kernel/trace/ftrace.c |   41 +++++++++++++++++++++++++++++++++++++++++
 1 files changed, 41 insertions(+), 0 deletions(-)
kernel/trace/ftrace.c +41 -0
index 2dae0c7..dc49ba6 100644
@@ -95,10 +95,13 @@ ftrace_func_t __ftrace_trace_function __read_mostly = ftrace_stub;
 ftrace_func_t ftrace_pid_function __read_mostly = ftrace_stub;
 static struct ftrace_ops global_ops;
 static struct ftrace_ops control_ops;
+static int non_global_ops_registered;
 
 static void
 ftrace_ops_list_func(unsigned long ip, unsigned long parent_ip);
 
+static void ftrace_graph_update_filter(void);
+
 /*
  * Traverse the ftrace_global_list, invoking all entries.  The reason that we
  * can use rcu_dereference_raw() is that elements removed from this list
@@ -330,6 +333,9 @@ static int __register_ftrace_function(struct ftrace_ops *ops)
 	if (!core_kernel_data((unsigned long)ops))
 		ops->flags |= FTRACE_OPS_FL_DYNAMIC;
 
+	if (!(ops->flags & FTRACE_OPS_FL_GLOBAL))
+		non_global_ops_registered++;
+
 	if (ops->flags & FTRACE_OPS_FL_GLOBAL) {
 		add_ftrace_ops(&ftrace_global_list, &global_ops, ops);
 		ops->flags |= FTRACE_OPS_FL_ENABLED;
@@ -359,6 +365,9 @@ static int __unregister_ftrace_function(struct ftrace_ops *ops)
 	if (FTRACE_WARN_ON(ops == &global_ops))
 		return -EINVAL;
 
+	if (!(ops->flags & FTRACE_OPS_FL_GLOBAL))
+		non_global_ops_registered--;
+
 	if (ops->flags & FTRACE_OPS_FL_GLOBAL) {
 		ret = remove_ftrace_ops(&ftrace_global_list, &global_ops, ops);
 		if (!ret)
@@ -1695,6 +1704,8 @@ static int __ftrace_modify_code(void *data)
 	if (*command & FTRACE_UPDATE_TRACE_FUNC)
 		ftrace_update_ftrace_func(ftrace_trace_function);
 
+	ftrace_graph_update_filter();
+
 	if (*command & FTRACE_START_FUNC_RET)
 		ftrace_enable_ftrace_graph_caller();
 	else if (*command & FTRACE_STOP_FUNC_RET)
@@ -4339,4 +4350,34 @@ void ftrace_graph_stop(void)
 {
 	ftrace_stop();
 }
+
+static trace_func_graph_ent_t ftrace_graph_entry_saved;
+
+int ftrace_graph_entry_filter(struct ftrace_graph_ent *ent)
+{
+	if (ftrace_ops_test(&global_ops, ent->func))
+		return ftrace_graph_entry_saved(ent);
+
+	return 0;
+}
+
+static void ftrace_graph_update_filter(void)
+{
+	bool installed = (ftrace_graph_entry == ftrace_graph_entry_filter);
+
+	if (!ftrace_graph_active)
+		return;
+
+	if (!installed && non_global_ops_registered) {
+		ftrace_graph_entry_saved = ftrace_graph_entry;
+		ftrace_graph_entry = ftrace_graph_entry_filter;
+		return;
+	}
+
+	if (installed && !non_global_ops_registered)
+		ftrace_graph_entry = ftrace_graph_entry_saved;
+}
+
+#else
+static void ftrace_graph_update_filter(void) { }
 #endif
Signature · 1 line
1.7.1
maintainer
#66 lore raw

Re: [PATCHv2 10/10] ftrace, graph: Add global_ops filter callback for graph tracing

Quoted 6 lines · Jiri Olsa

The function graph tracer should depend on the global_ops filter, and process only functions that pass the global_ops filter.

Currently the function graph tracer gets all the functions enabled for tracing no matter what ftrace_ops enabled them.

Quoted 6 lines

Adding a hook for the graph entry callback, which ensures the function is compared against the global_ops filter and bail out of if it does not match.

This hook is enabled only if there's at least one non global ftrace_ops registered.

Sorry, I don't like this fix. Right now just let function graph tracer act weird. If we put in this workaround, it would let us be lazy and not work on function graph for a proper fix.

The function graph code needs a overhaul anyway. A proper fix may require fixes in the arch code where the assembly is, as well as a bit of rewriting of the original code.

This is on my todo list, although it is a bit low priority now. With this patch set coming in, I can up the priority on the real fix. So please remove this patch, but keep pinging me to fix it for real ;)

-- Steve

author
#67 lore raw
In reply to Jiri Olsa

hi,
any feedback?

thanks,
jirka

Quoted 49 lines · Jiri Olsa

hi,
here's another version of perf support for function trace with filter. The changeset is working and hopefully is not introducing more bugs.. ;) still testing though..

It's still marked RFC since I'm not sure there's better way to go with patches 2 and 10. Also not sure if the condition fixed by patch 7 was not intentional. The rest is fixed/updated version of the v1 changes.

attached patches:
01/10 ftrace: Fix possible NULL dereferencing in __ftrace_hash_rec_update 02/10 ftrace: Change mcount call replacement logic
03/10 ftrace: Add enable/disable ftrace_ops control interface 04/10 ftrace, perf: Add open/close tracepoint perf registration actions 05/10 ftrace, perf: Add add/del tracepoint perf registration actions 06/10 ftrace, perf: Add support to use function tracepoint in perf 07/10 ftrace: Change filter/notrace set functions to return exit code 08/10 ftrace, perf: Distinguish ftrace function event field type 09/10 ftrace, perf: Add filter support for function trace event 10/10 ftrace, graph: Add global_ops filter callback for graph tracing

v2 changes:
01/10 - keeping the old fix instead of adding hash_has_contents func
I'll send separating patchset for this
02/10 - using different way to avoid the issue (3/9 in v1) 03/10 - using the way proposed by Steven for controling ftrace_ops

         (4/9 in v1)
 06/10 - added check ensuring the ftrace:function event could be used by
         root only (7/9 in v1)

08/10 - added more description (8/9 in v1)
09/10 - changed '&&' operator to '||' which seems more suitable
in this case (9/9 in v1)

thanks for comments,
jirka

---
 include/linux/ftrace.h             |   18 +++-
 include/linux/ftrace_event.h       |    9 +-
 include/linux/perf_event.h         |    3 +
 kernel/trace/ftrace.c              |  202 +++++++++++++++++++++++++++++------
 kernel/trace/trace.h               |   11 ++-
 kernel/trace/trace_event_perf.c    |  212 +++++++++++++++++++++++++++++-------
 kernel/trace/trace_events.c        |   12 ++-
 kernel/trace/trace_events_filter.c |  116 ++++++++++++++++++-
 kernel/trace/trace_export.c        |   53 ++++++++-
 kernel/trace/trace_kprobe.c        |    8 +-
 kernel/trace/trace_syscalls.c      |   18 +++-
 11 files changed, 562 insertions(+), 100 deletions(-)
maintainer
#68 lore raw
Quoted 3 lines · Jiri Olsa

hi,
any feedback?

Ah sorry, I think I missed this version. Too many other things going on at once.

-- Steve

#69 lore raw
In reply to Jiri Olsa
Quoted 2 lines · Jiri Olsa

hi,
any feedback?

Sorry for having lost track on this, will have a look soon.

Thanks.

Quoted 54 lines

thanks,
jirka

Quoted 49 lines · Jiri Olsa

hi,
here's another version of perf support for function trace with filter. The changeset is working and hopefully is not introducing more bugs.. ;) still testing though..

It's still marked RFC since I'm not sure there's better way to go with patches 2 and 10. Also not sure if the condition fixed by patch 7 was not intentional. The rest is fixed/updated version of the v1 changes.

attached patches:
01/10 ftrace: Fix possible NULL dereferencing in __ftrace_hash_rec_update 02/10 ftrace: Change mcount call replacement logic
03/10 ftrace: Add enable/disable ftrace_ops control interface 04/10 ftrace, perf: Add open/close tracepoint perf registration actions 05/10 ftrace, perf: Add add/del tracepoint perf registration actions 06/10 ftrace, perf: Add support to use function tracepoint in perf 07/10 ftrace: Change filter/notrace set functions to return exit code 08/10 ftrace, perf: Distinguish ftrace function event field type 09/10 ftrace, perf: Add filter support for function trace event 10/10 ftrace, graph: Add global_ops filter callback for graph tracing

v2 changes:
01/10 - keeping the old fix instead of adding hash_has_contents func
I'll send separating patchset for this
02/10 - using different way to avoid the issue (3/9 in v1) 03/10 - using the way proposed by Steven for controling ftrace_ops

         (4/9 in v1)
 06/10 - added check ensuring the ftrace:function event could be used by
         root only (7/9 in v1)

08/10 - added more description (8/9 in v1)
09/10 - changed '&&' operator to '||' which seems more suitable
in this case (9/9 in v1)

thanks for comments,
jirka

---
 include/linux/ftrace.h             |   18 +++-
 include/linux/ftrace_event.h       |    9 +-
 include/linux/perf_event.h         |    3 +
 kernel/trace/ftrace.c              |  202 +++++++++++++++++++++++++++++------
 kernel/trace/trace.h               |   11 ++-
 kernel/trace/trace_event_perf.c    |  212 +++++++++++++++++++++++++++++-------
 kernel/trace/trace_events.c        |   12 ++-
 kernel/trace/trace_events_filter.c |  116 ++++++++++++++++++-
 kernel/trace/trace_export.c        |   53 ++++++++-
 kernel/trace/trace_kprobe.c        |    8 +-
 kernel/trace/trace_syscalls.c      |   18 +++-
 11 files changed, 562 insertions(+), 100 deletions(-)
author
#70 lore raw

v3 cover ftrace, perf: Adding support to use function trace

In reply to Jiri Olsa

hi,
here's another version of perf support for function trace with filter.

attached patches:
1/8 ftrace: Change filter/notrace set functions to return exit code 2/8 ftrace: Fix possible NULL dereferencing in __ftrace_hash_rec_update 3/8 ftrace: Add enable/disable ftrace_ops control interface 4/8 ftrace, perf: Add open/close tracepoint perf registration actions 5/8 ftrace, perf: Add add/del tracepoint perf registration actions 6/8 ftrace, perf: Add support to use function tracepoint in perf 7/8 ftrace, perf: Distinguish ftrace function event field type 8/8 ftrace, perf: Add filter support for function trace event

v3 changes:
3/8 - renamed __add/remove_ftrace_ops
- fixed preemtp_enable/recursion_clear order in ftrace_ops_control_func
- renamed/commented API functions - enable/disable_ftrace_function

ommited graph tracer workarounf patch 10/10

v2 changes:
01/10 - keeping the old fix instead of adding hash_has_contents func
I'll send separating patchset for this
02/10 - using different way to avoid the issue (3/9 in v1) 03/10 - using the way proposed by Steven for controling ftrace_ops

         (4/9 in v1)
 06/10 - added check ensuring the ftrace:function event could be used by
         root only (7/9 in v1)

08/10 - added more description (8/9 in v1)
09/10 - changed '&&' operator to '||' which seems more suitable
in this case (9/9 in v1)

thanks,
jirka

---
 include/linux/ftrace.h             |   46 ++++++++-
 include/linux/ftrace_event.h       |    9 +-
 include/linux/perf_event.h         |    3 +
 kernel/trace/ftrace.c              |  135 ++++++++++++++++++++---
 kernel/trace/trace.h               |   11 ++-
 kernel/trace/trace_event_perf.c    |  212 +++++++++++++++++++++++++++++-------
 kernel/trace/trace_events.c        |   12 ++-
 kernel/trace/trace_events_filter.c |  116 ++++++++++++++++++-
 kernel/trace/trace_export.c        |   53 ++++++++-
 kernel/trace/trace_kprobe.c        |    8 +-
 kernel/trace/trace_syscalls.c      |   18 +++-
 11 files changed, 541 insertions(+), 82 deletions(-)
author
#71 lore raw

PATCH v3 1/8 ftrace: Change filter/notrace set functions to return exit code

Currently the ftrace_set_filter and ftrace_set_notrace functions do not return any return code. So there's no way for ftrace_ops user to tell wether the filter was correctly applied.

The set_ftrace_filter interface returns error in case the filter did not match:

  # echo krava > set_ftrace_filter
  bash: echo: write error: Invalid argument

Changing both ftrace_set_filter and ftrace_set_notrace functions to return zero if the filter was applied correctly or -E* values in case of error.

---
 include/linux/ftrace.h |    4 ++--
 kernel/trace/ftrace.c  |   16 ++++++++++------
 2 files changed, 12 insertions(+), 8 deletions(-)
include/linux/ftrace.h +2 -2
index 26eafce..523640f 100644
@@ -180,9 +180,9 @@ struct dyn_ftrace {
 };
 
 int ftrace_force_update(void);
-void ftrace_set_filter(struct ftrace_ops *ops, unsigned char *buf,
+int ftrace_set_filter(struct ftrace_ops *ops, unsigned char *buf,
 		       int len, int reset);
-void ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
+int ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
 			int len, int reset);
 void ftrace_set_global_filter(unsigned char *buf, int len, int reset);
 void ftrace_set_global_notrace(unsigned char *buf, int len, int reset);
kernel/trace/ftrace.c +10 -6
index 25b4f4d..09007c0 100644
@@ -2911,8 +2911,11 @@ ftrace_set_regex(struct ftrace_ops *ops, unsigned char *buf, int len,
 	mutex_lock(&ftrace_regex_lock);
 	if (reset)
 		ftrace_filter_reset(hash);
-	if (buf)
-		ftrace_match_records(hash, buf, len);
+	if (buf &&
+	    !ftrace_match_records(hash, buf, len)) {
+		ret = -EINVAL;
+		goto out_regex_unlock;
+	}
 
 	mutex_lock(&ftrace_lock);
 	ret = ftrace_hash_move(ops, enable, orig_hash, hash);
@@ -2922,6 +2925,7 @@ ftrace_set_regex(struct ftrace_ops *ops, unsigned char *buf, int len,
 
 	mutex_unlock(&ftrace_lock);
 
+ out_regex_unlock:
 	mutex_unlock(&ftrace_regex_lock);
 
 	free_ftrace_hash(hash);
@@ -2938,10 +2942,10 @@ ftrace_set_regex(struct ftrace_ops *ops, unsigned char *buf, int len,
  * Filters denote which functions should be enabled when tracing is enabled.
  * If @buf is NULL and reset is set, all functions will be enabled for tracing.
  */
-void ftrace_set_filter(struct ftrace_ops *ops, unsigned char *buf,
+int ftrace_set_filter(struct ftrace_ops *ops, unsigned char *buf,
 		       int len, int reset)
 {
-	ftrace_set_regex(ops, buf, len, reset, 1);
+	return ftrace_set_regex(ops, buf, len, reset, 1);
 }
 EXPORT_SYMBOL_GPL(ftrace_set_filter);
 
@@ -2956,10 +2960,10 @@ EXPORT_SYMBOL_GPL(ftrace_set_filter);
  * is enabled. If @buf is NULL and reset is set, all functions will be enabled
  * for tracing.
  */
-void ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
+int ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
 			int len, int reset)
 {
-	ftrace_set_regex(ops, buf, len, reset, 0);
+	return ftrace_set_regex(ops, buf, len, reset, 0);
 }
 EXPORT_SYMBOL_GPL(ftrace_set_notrace);
 /**
Signature · 1 line
1.7.1
author
#72 lore raw

PATCH v3 2/8 ftrace: Fix possible NULL dereferencing in __ftrace_hash_rec_update

In reply to Jiri Olsa

We need to check the existence of the other_hash before we touch its count variable.

This issue is hit only when non global ftrace_ops is used. The global ftrace_ops is initialized with empty hashes.

---
 kernel/trace/ftrace.c |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)
kernel/trace/ftrace.c +2 -1
index 09007c0..7eb702f 100644
@@ -1372,7 +1372,8 @@ static void __ftrace_hash_rec_update(struct ftrace_ops *ops,
 			if (filter_hash && in_hash && !in_other_hash)
 				match = 1;
 			else if (!filter_hash && in_hash &&
-				 (in_other_hash || !other_hash->count))
+				 (in_other_hash ||
+				  !other_hash || !other_hash->count))
 				match = 1;
 		}
 		if (!match)
Signature · 1 line
1.7.1
maintainer
#73 lore raw

Re: [PATCH 2/8] ftrace: Fix possible NULL dereferencing in __ftrace_hash_rec_update

Quoted 6 lines · Jiri Olsa

We need to check the existence of the other_hash before we touch its count variable.

This issue is hit only when non global ftrace_ops is used. The global ftrace_ops is initialized with empty hashes.

As this wasn't the first time this bug showed up, I fixed this by this patch:

https://lkml.org/lkml/2011/12/21/197

So you can remove this patch as well.

-- Steve

Quoted 19 lines
---
 kernel/trace/ftrace.c |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)
kernel/trace/ftrace.c +2 -1
index 09007c0..7eb702f 100644
@@ -1372,7 +1372,8 @@ static void __ftrace_hash_rec_update(struct ftrace_ops *ops,
 			if (filter_hash && in_hash && !in_other_hash)
 				match = 1;
 			else if (!filter_hash && in_hash &&
-				 (in_other_hash || !other_hash->count))
+				 (in_other_hash ||
+				  !other_hash || !other_hash->count))
 				match = 1;
 		}
 		if (!match)
author
#74 lore raw

PATCH v3 3/8 ftrace: Add enable/disable ftrace_ops control interface

In reply to Jiri Olsa

Adding a way to temporarily enable/disable ftrace_ops. The change follows the same way as 'global' ftrace_ops are done.

Introducing 2 global ftrace_ops - control_ops and ftrace_control_list which take over all ftrace_ops registered with FTRACE_OPS_FL_CONTROL flag. In addition new per cpu flag called 'disabled' is also added to ftrace_ops to provide the control information for each cpu.

When ftrace_ops with FTRACE_OPS_FL_CONTROL is registered, it is set as disabled for all cpus.

The ftrace_control_list contains all the registered 'control' ftrace_ops. The control_ops provides function which iterates ftrace_control_list and does the check for 'disabled' flag on current cpu.

Adding 2 inline functions ftrace_function_enable/ftrace_function_disable, which enable/disable the ftrace_ops for given cpu.

---
 include/linux/ftrace.h |   42 +++++++++++++++++
 kernel/trace/ftrace.c  |  116 +++++++++++++++++++++++++++++++++++++++++++++---
 kernel/trace/trace.h   |    2 +
 3 files changed, 153 insertions(+), 7 deletions(-)
include/linux/ftrace.h +42 -0
index 523640f..67b8236 100644
@@ -35,12 +35,14 @@ enum {
 	FTRACE_OPS_FL_ENABLED		= 1 << 0,
 	FTRACE_OPS_FL_GLOBAL		= 1 << 1,
 	FTRACE_OPS_FL_DYNAMIC		= 1 << 2,
+	FTRACE_OPS_FL_CONTROL		= 1 << 3,
 };
 
 struct ftrace_ops {
 	ftrace_func_t			func;
 	struct ftrace_ops		*next;
 	unsigned long			flags;
+	void __percpu			*disabled;
 #ifdef CONFIG_DYNAMIC_FTRACE
 	struct ftrace_hash		*notrace_hash;
 	struct ftrace_hash		*filter_hash;
@@ -97,6 +99,46 @@ int register_ftrace_function(struct ftrace_ops *ops);
 int unregister_ftrace_function(struct ftrace_ops *ops);
 void clear_ftrace_function(void);
 
+/**
+ * ftrace_function_enable - enable controlled ftrace_ops on given cpu
+ *
+ * This function enables tracing on given cpu by decreasing
+ * the per cpu control variable.
+ * It must be called with preemption disabled and only on
+ * ftrace_ops registered with FTRACE_OPS_FL_CONTROL.
+ */
+static inline void ftrace_function_enable(struct ftrace_ops *ops, int cpu)
+{
+	atomic_t *disabled;
+
+	if (WARN_ON_ONCE(!(ops->flags & FTRACE_OPS_FL_CONTROL)) ||
+			 !preempt_count())
+		return;
+
+	disabled = per_cpu_ptr(ops->disabled, cpu);
+	atomic_dec(disabled);
+}
+
+/**
+ * ftrace_function_disable - enable controlled ftrace_ops on given cpu
+ *
+ * This function enables tracing on given cpu by decreasing
+ * the per cpu control variable.
+ * It must be called with preemption disabled and only on
+ * ftrace_ops registered with FTRACE_OPS_FL_CONTROL.
+ */
+static inline void ftrace_function_disable(struct ftrace_ops *ops, int cpu)
+{
+	atomic_t *disabled;
+
+	if (WARN_ON_ONCE(!(ops->flags & FTRACE_OPS_FL_CONTROL)) ||
+			 !preempt_count())
+		return;
+
+	disabled = per_cpu_ptr(ops->disabled, cpu);
+	atomic_inc(disabled);
+}
+
 extern void ftrace_stub(unsigned long a0, unsigned long a1);
 
 #else /* !CONFIG_FUNCTION_TRACER */
kernel/trace/ftrace.c +109 -7

171 lines not on this page · raw message

kernel/trace/trace.h +2 -0
index 2c26574..41c54e3 100644
@@ -288,6 +288,8 @@ struct tracer {
 /* for function tracing recursion */
 #define TRACE_INTERNAL_BIT		(1<<11)
 #define TRACE_GLOBAL_BIT		(1<<12)
+#define TRACE_CONTROL_BIT		(1<<13)
+
 /*
  * Abuse of the trace_recursion.
  * As we need a way to maintain state if we are tracing the function
Signature · 1 line
1.7.1
maintainer
#75 lore raw

Re: [PATCH 3/8] ftrace: Add enable/disable ftrace_ops control interface

Quoted 52 lines · Jiri Olsa

Adding a way to temporarily enable/disable ftrace_ops. The change follows the same way as 'global' ftrace_ops are done.

Introducing 2 global ftrace_ops - control_ops and ftrace_control_list which take over all ftrace_ops registered with FTRACE_OPS_FL_CONTROL flag. In addition new per cpu flag called 'disabled' is also added to ftrace_ops to provide the control information for each cpu.

When ftrace_ops with FTRACE_OPS_FL_CONTROL is registered, it is set as disabled for all cpus.

The ftrace_control_list contains all the registered 'control' ftrace_ops. The control_ops provides function which iterates ftrace_control_list and does the check for 'disabled' flag on current cpu.

Adding 2 inline functions ftrace_function_enable/ftrace_function_disable, which enable/disable the ftrace_ops for given cpu.

---
 include/linux/ftrace.h |   42 +++++++++++++++++
 kernel/trace/ftrace.c  |  116 +++++++++++++++++++++++++++++++++++++++++++++---
 kernel/trace/trace.h   |    2 +
 3 files changed, 153 insertions(+), 7 deletions(-)
include/linux/ftrace.h +6 -0
index 523640f..67b8236 100644
@@ -35,12 +35,14 @@ enum {
 	FTRACE_OPS_FL_ENABLED		= 1 << 0,
 	FTRACE_OPS_FL_GLOBAL		= 1 << 1,
 	FTRACE_OPS_FL_DYNAMIC		= 1 << 2,
+	FTRACE_OPS_FL_CONTROL		= 1 << 3,
 };
 
 struct ftrace_ops {
 	ftrace_func_t			func;
 	struct ftrace_ops		*next;
 	unsigned long			flags;
+	void __percpu			*disabled;
 #ifdef CONFIG_DYNAMIC_FTRACE
 	struct ftrace_hash		*notrace_hash;
 	struct ftrace_hash		*filter_hash;
@@ -97,6 +99,46 @@ int register_ftrace_function(struct ftrace_ops *ops);
 int unregister_ftrace_function(struct ftrace_ops *ops);
 void clear_ftrace_function(void);
 
+/**
+ * ftrace_function_enable - enable controlled ftrace_ops on given cpu
+ *
+ * This function enables tracing on given cpu by decreasing

include/linux/ftrace.h int register_ftrace_function(struct ftrace_ops *ops); · line 115

Quoted 10 lines
include/linux/ftrace.h +10 -0
+ * the per cpu control variable.
+ * It must be called with preemption disabled and only on
+ * ftrace_ops registered with FTRACE_OPS_FL_CONTROL.
+ */
+static inline void ftrace_function_enable(struct ftrace_ops *ops, int cpu)
+{
+	atomic_t *disabled;
+
+	if (WARN_ON_ONCE(!(ops->flags & FTRACE_OPS_FL_CONTROL)) ||
+			 !preempt_count())

The WARN_ON_ONCE() should also include the !preempt_count().

Quoted 10 lines
include/linux/ftrace.h +10 -0
+		return;
+
+	disabled = per_cpu_ptr(ops->disabled, cpu);
+	atomic_dec(disabled);
+}
+
+/**
+ * ftrace_function_disable - enable controlled ftrace_ops on given cpu
+ *
+ * This function enables tracing on given cpu by decreasing

include/linux/ftrace.h int register_ftrace_function(struct ftrace_ops *ops); · line 135

Quoted 10 lines
include/linux/ftrace.h +10 -0
+ * the per cpu control variable.
+ * It must be called with preemption disabled and only on
+ * ftrace_ops registered with FTRACE_OPS_FL_CONTROL.
+ */
+static inline void ftrace_function_disable(struct ftrace_ops *ops, int cpu)
+{
+	atomic_t *disabled;
+
+	if (WARN_ON_ONCE(!(ops->flags & FTRACE_OPS_FL_CONTROL)) ||
+			 !preempt_count())

Same here.

Quoted 60 lines
include/linux/ftrace.h +6 -0
+		return;
+
+	disabled = per_cpu_ptr(ops->disabled, cpu);
+	atomic_inc(disabled);
+}
+
 extern void ftrace_stub(unsigned long a0, unsigned long a1);
 
 #else /* !CONFIG_FUNCTION_TRACER */
kernel/trace/ftrace.c +23 -0
index 7eb702f..1b56013 100644
@@ -60,6 +60,8 @@
 #define FTRACE_HASH_DEFAULT_BITS 10
 #define FTRACE_HASH_MAX_BITS 12
 
+#define FL_GLOBAL_CONTROL (FTRACE_OPS_FL_GLOBAL | FTRACE_OPS_FL_CONTROL)
+
 /* ftrace_enabled is a method to turn ftrace on or off */
 int ftrace_enabled __read_mostly;
 static int last_ftrace_enabled;
@@ -87,12 +89,14 @@ static struct ftrace_ops ftrace_list_end __read_mostly = {
 };
 
 static struct ftrace_ops *ftrace_global_list __read_mostly = &ftrace_list_end;
+static struct ftrace_ops *ftrace_control_list __read_mostly = &ftrace_list_end;
 static struct ftrace_ops *ftrace_ops_list __read_mostly = &ftrace_list_end;
 ftrace_func_t ftrace_trace_function __read_mostly = ftrace_stub;
 static ftrace_func_t __ftrace_trace_function_delay __read_mostly = ftrace_stub;
 ftrace_func_t __ftrace_trace_function __read_mostly = ftrace_stub;
 ftrace_func_t ftrace_pid_function __read_mostly = ftrace_stub;
 static struct ftrace_ops global_ops;
+static struct ftrace_ops control_ops;
 
 static void
 ftrace_ops_list_func(unsigned long ip, unsigned long parent_ip);
@@ -166,6 +170,38 @@ static void ftrace_test_stop_func(unsigned long ip, unsigned long parent_ip)
 }
 #endif
 
+static void control_ops_disable_all(struct ftrace_ops *ops)
+{
+	int cpu;
+
+	for_each_possible_cpu(cpu)
+		atomic_set(per_cpu_ptr(ops->disabled, cpu), 1);
+}
+
+static int control_ops_alloc(struct ftrace_ops *ops)
+{
+	atomic_t *disabled;
+
+	disabled = alloc_percpu(atomic_t);
+	if (!disabled)
+		return -ENOMEM;
+
+	ops->disabled = disabled;
+	control_ops_disable_all(ops);
+	return 0;

kernel/trace/ftrace.c static void ftrace_test_stop_func(unsigned long ip, unsigned long parent_ip) · line 201

Quoted 10 lines
kernel/trace/ftrace.c +10 -0
+}
+
+static void control_ops_free(struct ftrace_ops *ops)
+{
+	free_percpu(ops->disabled);
+}
+
+static int control_ops_is_disabled(struct ftrace_ops *ops)
+{
+	atomic_t *disabled = this_cpu_ptr(ops->disabled);

Again, the use of "this_cpu_ptr" is wrong. Gah! We should nuke all of that crap.

Quoted 34 lines
kernel/trace/ftrace.c +23 -0
+	return atomic_read(disabled);
+}
+
 static void update_global_ops(void)
 {
 	ftrace_func_t func;
@@ -257,6 +293,26 @@ static int remove_ftrace_ops(struct ftrace_ops **list, struct ftrace_ops *ops)
 	return 0;
 }
 
+static void add_ftrace_list_ops(struct ftrace_ops **list,
+				struct ftrace_ops *main_ops,
+				struct ftrace_ops *ops)
+{
+	int first = *list == &ftrace_list_end;
+	add_ftrace_ops(list, ops);
+	if (first)
+		add_ftrace_ops(&ftrace_ops_list, main_ops);
+}
+
+static int remove_ftrace_list_ops(struct ftrace_ops **list,
+				  struct ftrace_ops *main_ops,
+				  struct ftrace_ops *ops)
+{
+	int ret = remove_ftrace_ops(list, ops);
+	if (!ret && *list == &ftrace_list_end)
+		ret = remove_ftrace_ops(&ftrace_ops_list, main_ops);
+	return ret;
+}
+
 static int __register_ftrace_function(struct ftrace_ops *ops)
 {
 	if (ftrace_disabled)

kernel/trace/ftrace.c static int __register_ftrace_function(struct ftrace_ops *ops) · line 327

Quoted 4 lines
kernel/trace/ftrace.c +1 -0
@@ -268,15 +324,19 @@ static int __register_ftrace_function(struct ftrace_ops *ops)
 	if (WARN_ON(ops->flags & FTRACE_OPS_FL_ENABLED))
 		return -EBUSY;
 
+	if ((ops->flags & FL_GLOBAL_CONTROL) == FL_GLOBAL_CONTROL)

No biggy, but I usually find:

	if (ops->flags & FL_GLOBAL_CONTROL)

more readable. With what you have, I looked at that condition three times to figure out what was different between what was '&'d with the flags and what was being equal too. Usually the ((flags & X) == Y) is done to check if a subset of bits are set within a mask of bits.

kernel/trace/ftrace.c static int __register_ftrace_function(struct ftrace_ops *ops) · line 334

Quoted 9 lines
kernel/trace/ftrace.c +3 -2
+		return -EINVAL;
+
 	if (!core_kernel_data((unsigned long)ops))
 		ops->flags |= FTRACE_OPS_FL_DYNAMIC;
 
 	if (ops->flags & FTRACE_OPS_FL_GLOBAL) {
-		int first = ftrace_global_list == &ftrace_list_end;
-		add_ftrace_ops(&ftrace_global_list, ops);
+		add_ftrace_list_ops(&ftrace_global_list, &global_ops, ops);

Much better and easier to read :)

kernel/trace/ftrace.c static int __register_ftrace_function(struct ftrace_ops *ops) · line 342

Quoted 10 lines
kernel/trace/ftrace.c +4 -2
 		ops->flags |= FTRACE_OPS_FL_ENABLED;
-		if (first)
-			add_ftrace_ops(&ftrace_ops_list, &global_ops);
+	} else if (ops->flags & FTRACE_OPS_FL_CONTROL) {
+		if (control_ops_alloc(ops))
+			return -ENOMEM;
+		add_ftrace_list_ops(&ftrace_control_list, &control_ops, ops);
 	} else
 		add_ftrace_ops(&ftrace_ops_list, ops);
 

The rest looks good.

-- Steve

author
#76 lore raw

Re: [PATCH 3/8] ftrace: Add enable/disable ftrace_ops control interface

Quoted 1 line · Steven Rostedt

On Wed, 2011-12-21 at 12:48 +0100, Jiri Olsa wrote:

SNIP

Quoted 8 lines
Quoted 8 lines

+/**
+ * ftrace_function_enable - enable controlled ftrace_ops on given cpu
+ *
+ * This function enables tracing on given cpu by decreasing
+ * the per cpu control variable.
+ * It must be called with preemption disabled and only on
+ * ftrace_ops registered with FTRACE_OPS_FL_CONTROL.
+ */

Quoted 9 lines

include/linux/ftrace.h int register_ftrace_function(struct ftrace_ops *ops); · line 115

Quoted 6 lines
include/linux/ftrace.h +6 -0
+static inline void ftrace_function_enable(struct ftrace_ops *ops, int cpu)
+{
+	atomic_t *disabled;
+
+	if (WARN_ON_ONCE(!(ops->flags & FTRACE_OPS_FL_CONTROL)) ||
+			 !preempt_count())

The WARN_ON_ONCE() should also include the !preempt_count().

ouch, that was initial intention.. need eye doctor ;)

Quoted 7 lines
Quoted 7 lines
include/linux/ftrace.h +6 -0
+		return;
+
+	disabled = per_cpu_ptr(ops->disabled, cpu);
+	atomic_dec(disabled);
+}
+

SNIP

Quoted 7 lines

kernel/trace/ftrace.c static void ftrace_test_stop_func(unsigned long ip, unsigned long parent_ip) · line 201

Quoted 4 lines
kernel/trace/ftrace.c +4 -0
+
+static int control_ops_is_disabled(struct ftrace_ops *ops)
+{
+	atomic_t *disabled = this_cpu_ptr(ops->disabled);

Again, the use of "this_cpu_ptr" is wrong. Gah! We should nuke all of that crap.

will nuke this one..

Quoted 14 lines

kernel/trace/ftrace.c static int __register_ftrace_function(struct ftrace_ops *ops) · line 327

Quoted 11 lines

static int __register_ftrace_function(struct ftrace_ops *ops) {

kernel/trace/ftrace.c +1 -0
 	if (ftrace_disabled)
@@ -268,15 +324,19 @@ static int __register_ftrace_function(struct ftrace_ops *ops)
 	if (WARN_ON(ops->flags & FTRACE_OPS_FL_ENABLED))
 		return -EBUSY;
 
+	if ((ops->flags & FL_GLOBAL_CONTROL) == FL_GLOBAL_CONTROL)

No biggy, but I usually find:

Quoted 6 lines
	if (ops->flags & FL_GLOBAL_CONTROL)

more readable. With what you have, I looked at that condition three times to figure out what was different between what was '&'d with the flags and what was being equal too. Usually the ((flags & X) == Y) is done to check if a subset of bits are set within a mask of bits.

Well, thats what I need to do here. Bail out if both bits are set, since we dont support both global and control flags set at the same time.. I'll add some comment to it.

thanks,
jirka

maintainer
#77 lore raw

Re: [PATCH 3/8] ftrace: Add enable/disable ftrace_ops control interface

Quoted 9 lines · Jiri Olsa
Quoted 4 lines

more readable. With what you have, I looked at that condition three times to figure out what was different between what was '&'d with the flags and what was being equal too. Usually the ((flags & X) == Y) is done to check if a subset of bits are set within a mask of bits.

Well, thats what I need to do here. Bail out if both bits are set, since we dont support both global and control flags set at the same time.. I'll add some comment to it.

Ah that's right, that's not a single bit. OK, you need to rename FL_GLOBAL_CONTROL to FL_GLOBAL_CONTROL_BITS or _MASK. _MASK may be better, as I think it's used more often for this case.

Otherwise it looks like a single bit. Then you wont even need the comment. But I wont stop you from adding one.

-- Steve

#78 lore raw

Re: [PATCH 3/8] ftrace: Add enable/disable ftrace_ops control interface

In reply to Steven Rostedt
Quoted 8 lines · Steven Rostedt

kernel/trace/ftrace.c static void ftrace_test_stop_func(unsigned long ip, unsigned long parent_ip) · line 201

Quoted 3 lines · Jiri Olsa
kernel/trace/ftrace.c +3 -0
+static int control_ops_is_disabled(struct ftrace_ops *ops)
+{
+	atomic_t *disabled = this_cpu_ptr(ops->disabled);

Again, the use of "this_cpu_ptr" is wrong. Gah! We should nuke all of that crap.

Is it? It includes the preemption check if CONFIG_DEBUG_PREEMPT, just like __get_cpu_var()

Just saying that because in a later version of this patch, Jiri used per_cpu_ptr(ops->disabled, cpu). And this is the wrong thing to do given that we always fetch the local pointer and per_cpu_ptr() doesn't check for preemption disabled.

Quoted 2 lines
Quoted 2 lines
kernel/trace/ftrace.c +2 -0
+	return atomic_read(disabled);
+}
author
#79 lore raw

PATCH v3 4/8 ftrace, perf: Add open/close tracepoint perf registration actions

In reply to Jiri Olsa

Adding TRACE_REG_PERF_OPEN and TRACE_REG_PERF_CLOSE to differentiate register/unregister from open/close actions.

The register/unregister actions are invoked for the first/last tracepoint user when opening/closing the evet.

The open/close actions are invoked for each tracepoint user when opening/closing the event.

---
 include/linux/ftrace_event.h    |    6 +-
 kernel/trace/trace.h            |    5 ++
 kernel/trace/trace_event_perf.c |  116 +++++++++++++++++++++++++--------------
 kernel/trace/trace_events.c     |   10 ++-
 kernel/trace/trace_kprobe.c     |    6 ++-
 kernel/trace/trace_syscalls.c   |   14 +++-
 6 files changed, 106 insertions(+), 51 deletions(-)
include/linux/ftrace_event.h +4 -2
index c3da42d..195e360 100644
@@ -146,6 +146,8 @@ enum trace_reg {
 	TRACE_REG_UNREGISTER,
 	TRACE_REG_PERF_REGISTER,
 	TRACE_REG_PERF_UNREGISTER,
+	TRACE_REG_PERF_OPEN,
+	TRACE_REG_PERF_CLOSE,
 };
 
 struct ftrace_event_call;
@@ -157,7 +159,7 @@ struct ftrace_event_class {
 	void			*perf_probe;
 #endif
 	int			(*reg)(struct ftrace_event_call *event,
-				       enum trace_reg type);
+				       enum trace_reg type, void *data);
 	int			(*define_fields)(struct ftrace_event_call *);
 	struct list_head	*(*get_fields)(struct ftrace_event_call *);
 	struct list_head	fields;
@@ -165,7 +167,7 @@ struct ftrace_event_class {
 };
 
 extern int ftrace_event_reg(struct ftrace_event_call *event,
-			    enum trace_reg type);
+			    enum trace_reg type, void *data);
 
 enum {
 	TRACE_EVENT_FL_ENABLED_BIT,
kernel/trace/trace.h +5 -0
index 41c54e3..85732a8 100644
@@ -828,4 +828,9 @@ extern const char *__stop___trace_bprintk_fmt[];
 	FTRACE_ENTRY(call, struct_name, id, PARAMS(tstruct), PARAMS(print))
 #include "trace_entries.h"
 
+#ifdef CONFIG_PERF_EVENTS
+int perf_ftrace_event_register(struct ftrace_event_call *call,
+			       enum trace_reg type, void *data);
+#endif
+
 #endif /* _LINUX_KERNEL_TRACE_H */
kernel/trace/trace_event_perf.c +75 -41

154 lines not on this page · raw message

kernel/trace/trace_events.c +7 -3
index c212a7f..5138fea 100644
@@ -147,7 +147,8 @@ int trace_event_raw_init(struct ftrace_event_call *call)
 }
 EXPORT_SYMBOL_GPL(trace_event_raw_init);
 
-int ftrace_event_reg(struct ftrace_event_call *call, enum trace_reg type)
+int ftrace_event_reg(struct ftrace_event_call *call,
+		     enum trace_reg type, void *data)
 {
 	switch (type) {
 	case TRACE_REG_REGISTER:
@@ -170,6 +171,9 @@ int ftrace_event_reg(struct ftrace_event_call *call, enum trace_reg type)
 					    call->class->perf_probe,
 					    call);
 		return 0;
+	case TRACE_REG_PERF_OPEN:
+	case TRACE_REG_PERF_CLOSE:
+		return 0;
 #endif
 	}
 	return 0;
@@ -209,7 +213,7 @@ static int ftrace_event_enable_disable(struct ftrace_event_call *call,
 				tracing_stop_cmdline_record();
 				call->flags &= ~TRACE_EVENT_FL_RECORDED_CMD;
 			}
-			call->class->reg(call, TRACE_REG_UNREGISTER);
+			call->class->reg(call, TRACE_REG_UNREGISTER, NULL);
 		}
 		break;
 	case 1:
@@ -218,7 +222,7 @@ static int ftrace_event_enable_disable(struct ftrace_event_call *call,
 				tracing_start_cmdline_record();
 				call->flags |= TRACE_EVENT_FL_RECORDED_CMD;
 			}
-			ret = call->class->reg(call, TRACE_REG_REGISTER);
+			ret = call->class->reg(call, TRACE_REG_REGISTER, NULL);
 			if (ret) {
 				tracing_stop_cmdline_record();
 				pr_info("event trace: Could not enable event "
kernel/trace/trace_kprobe.c +5 -1
index 00d527c..5667f89 100644
@@ -1892,7 +1892,8 @@ static __kprobes void kretprobe_perf_func(struct kretprobe_instance *ri,
 #endif	/* CONFIG_PERF_EVENTS */
 
 static __kprobes
-int kprobe_register(struct ftrace_event_call *event, enum trace_reg type)
+int kprobe_register(struct ftrace_event_call *event,
+		    enum trace_reg type, void *data)
 {
 	struct trace_probe *tp = (struct trace_probe *)event->data;
 
@@ -1909,6 +1910,9 @@ int kprobe_register(struct ftrace_event_call *event, enum trace_reg type)
 	case TRACE_REG_PERF_UNREGISTER:
 		disable_trace_probe(tp, TP_FLAG_PROFILE);
 		return 0;
+	case TRACE_REG_PERF_OPEN:
+	case TRACE_REG_PERF_CLOSE:
+		return 0;
 #endif
 	}
 	return 0;
kernel/trace/trace_syscalls.c +10 -4
index 5f35f6f..8599c1d 100644
@@ -18,9 +18,9 @@ static DECLARE_BITMAP(enabled_enter_syscalls, NR_syscalls);
 static DECLARE_BITMAP(enabled_exit_syscalls, NR_syscalls);
 
 static int syscall_enter_register(struct ftrace_event_call *event,
-				 enum trace_reg type);
+				 enum trace_reg type, void *data);
 static int syscall_exit_register(struct ftrace_event_call *event,
-				 enum trace_reg type);
+				 enum trace_reg type, void *data);
 
 static int syscall_enter_define_fields(struct ftrace_event_call *call);
 static int syscall_exit_define_fields(struct ftrace_event_call *call);
@@ -650,7 +650,7 @@ void perf_sysexit_disable(struct ftrace_event_call *call)
 #endif /* CONFIG_PERF_EVENTS */
 
 static int syscall_enter_register(struct ftrace_event_call *event,
-				 enum trace_reg type)
+				 enum trace_reg type, void *data)
 {
 	switch (type) {
 	case TRACE_REG_REGISTER:
@@ -665,13 +665,16 @@ static int syscall_enter_register(struct ftrace_event_call *event,
 	case TRACE_REG_PERF_UNREGISTER:
 		perf_sysenter_disable(event);
 		return 0;
+	case TRACE_REG_PERF_OPEN:
+	case TRACE_REG_PERF_CLOSE:
+		return 0;
 #endif
 	}
 	return 0;
 }
 
 static int syscall_exit_register(struct ftrace_event_call *event,
-				 enum trace_reg type)
+				 enum trace_reg type, void *data)
 {
 	switch (type) {
 	case TRACE_REG_REGISTER:
@@ -686,6 +689,9 @@ static int syscall_exit_register(struct ftrace_event_call *event,
 	case TRACE_REG_PERF_UNREGISTER:
 		perf_sysexit_disable(event);
 		return 0;
+	case TRACE_REG_PERF_OPEN:
+	case TRACE_REG_PERF_CLOSE:
+		return 0;
 #endif
 	}
 	return 0;
Signature · 1 line
1.7.1
author
#80 lore raw

PATCH v3 5/8 ftrace, perf: Add add/del tracepoint perf registration actions

In reply to Jiri Olsa

Adding TRACE_REG_PERF_ADD and TRACE_REG_PERF_DEL to handle perf event schedule in/out actions.

The add action is invoked for when the perf event is scheduled in, while the del action is invoked when the event is scheduled out.

---
 include/linux/ftrace_event.h    |    2 ++
 kernel/trace/trace_event_perf.c |    4 +++-
 kernel/trace/trace_events.c     |    2 ++
 kernel/trace/trace_kprobe.c     |    2 ++
 kernel/trace/trace_syscalls.c   |    4 ++++
 5 files changed, 13 insertions(+), 1 deletions(-)
include/linux/ftrace_event.h +2 -0
index 195e360..2bf677c 100644
@@ -148,6 +148,8 @@ enum trace_reg {
 	TRACE_REG_PERF_UNREGISTER,
 	TRACE_REG_PERF_OPEN,
 	TRACE_REG_PERF_CLOSE,
+	TRACE_REG_PERF_ADD,
+	TRACE_REG_PERF_DEL,
 };
 
 struct ftrace_event_call;
kernel/trace/trace_event_perf.c +3 -1
index 0cfcc37..d72af0b 100644
@@ -211,12 +211,14 @@ int perf_trace_add(struct perf_event *p_event, int flags)
 	list = this_cpu_ptr(pcpu_list);
 	hlist_add_head_rcu(&p_event->hlist_entry, list);
 
-	return 0;
+	return tp_event->class->reg(tp_event, TRACE_REG_PERF_ADD, p_event);
 }
 
 void perf_trace_del(struct perf_event *p_event, int flags)
 {
+	struct ftrace_event_call *tp_event = p_event->tp_event;
 	hlist_del_rcu(&p_event->hlist_entry);
+	tp_event->class->reg(tp_event, TRACE_REG_PERF_DEL, p_event);
 }
 
 __kprobes void *perf_trace_buf_prepare(int size, unsigned short type,
kernel/trace/trace_events.c +2 -0
index 5138fea..079a93a 100644
@@ -173,6 +173,8 @@ int ftrace_event_reg(struct ftrace_event_call *call,
 		return 0;
 	case TRACE_REG_PERF_OPEN:
 	case TRACE_REG_PERF_CLOSE:
+	case TRACE_REG_PERF_ADD:
+	case TRACE_REG_PERF_DEL:
 		return 0;
 #endif
 	}
kernel/trace/trace_kprobe.c +2 -0
index 5667f89..580a05e 100644
@@ -1912,6 +1912,8 @@ int kprobe_register(struct ftrace_event_call *event,
 		return 0;
 	case TRACE_REG_PERF_OPEN:
 	case TRACE_REG_PERF_CLOSE:
+	case TRACE_REG_PERF_ADD:
+	case TRACE_REG_PERF_DEL:
 		return 0;
 #endif
 	}
kernel/trace/trace_syscalls.c +4 -0
index 8599c1d..5e4f62e 100644
@@ -667,6 +667,8 @@ static int syscall_enter_register(struct ftrace_event_call *event,
 		return 0;
 	case TRACE_REG_PERF_OPEN:
 	case TRACE_REG_PERF_CLOSE:
+	case TRACE_REG_PERF_ADD:
+	case TRACE_REG_PERF_DEL:
 		return 0;
 #endif
 	}
@@ -691,6 +693,8 @@ static int syscall_exit_register(struct ftrace_event_call *event,
 		return 0;
 	case TRACE_REG_PERF_OPEN:
 	case TRACE_REG_PERF_CLOSE:
+	case TRACE_REG_PERF_ADD:
+	case TRACE_REG_PERF_DEL:
 		return 0;
 #endif
 	}
Signature · 1 line
1.7.1
author
#81 lore raw

PATCH v3 6/8 ftrace, perf: Add support to use function tracepoint in perf

In reply to Jiri Olsa

Adding perf registration support for the ftrace function event, so it is now possible to register it via perf interface.

The perf_event struct statically contains ftrace_ops as a handle for function tracer. The function tracer is registered/unregistered in open/close actions, and enabled/disabled in add/del actions.

It is now possible to use function trace within perf commands like:

perf record -e ftrace:function ls
perf stat -e ftrace:function ls

Allowed only for root.

---
 include/linux/perf_event.h      |    3 +
 kernel/trace/trace.h            |    2 +
 kernel/trace/trace_event_perf.c |   92 +++++++++++++++++++++++++++++++++++++++
 kernel/trace/trace_export.c     |   28 ++++++++++++
 4 files changed, 125 insertions(+), 0 deletions(-)
include/linux/perf_event.h +3 -0
index 564769c..47a9df6 100644
@@ -848,6 +848,9 @@ struct perf_event {
 #ifdef CONFIG_EVENT_TRACING
 	struct ftrace_event_call	*tp_event;
 	struct event_filter		*filter;
+#ifdef CONFIG_FUNCTION_TRACER
+	struct ftrace_ops               ftrace_ops;
+#endif
 #endif
 
 #ifdef CONFIG_CGROUP_PERF
kernel/trace/trace.h +2 -0
index 85732a8..e88e58a 100644
@@ -591,6 +591,8 @@ static inline int ftrace_trace_task(struct task_struct *task)
 static inline int ftrace_is_dead(void) { return 0; }
 #endif
 
+int ftrace_event_is_function(struct ftrace_event_call *call);
+
 /*
  * struct trace_parser - servers for reading the user input separated by spaces
  * @cont: set if the input is not complete - no final space char was found
kernel/trace/trace_event_perf.c +92 -0

101 lines not on this page · raw message

kernel/trace/trace_export.c +28 -0
index bbeec31..867653c 100644
@@ -131,6 +131,28 @@ ftrace_define_fields_##name(struct ftrace_event_call *event_call)	\
 
 #include "trace_entries.h"
 
+static int ftrace_event_class_register(struct ftrace_event_call *call,
+				       enum trace_reg type, void *data)
+{
+	switch (type) {
+	case TRACE_REG_PERF_REGISTER:
+	case TRACE_REG_PERF_UNREGISTER:
+		return 0;
+	case TRACE_REG_PERF_OPEN:
+	case TRACE_REG_PERF_CLOSE:
+	case TRACE_REG_PERF_ADD:
+	case TRACE_REG_PERF_DEL:
+#ifdef CONFIG_PERF_EVENTS
+		return perf_ftrace_event_register(call, type, data);
+#endif
+	case TRACE_REG_REGISTER:
+	case TRACE_REG_UNREGISTER:
+		break;
+	}
+
+	return -EINVAL;
+}
+
 #undef __entry
 #define __entry REC
 
@@ -159,6 +181,7 @@ struct ftrace_event_class event_class_ftrace_##call = {			\
 	.system			= __stringify(TRACE_SYSTEM),		\
 	.define_fields		= ftrace_define_fields_##call,		\
 	.fields			= LIST_HEAD_INIT(event_class_ftrace_##call.fields),\
+	.reg			= ftrace_event_class_register,		\
 };									\
 									\
 struct ftrace_event_call __used event_##call = {			\
@@ -170,4 +193,9 @@ struct ftrace_event_call __used event_##call = {			\
 struct ftrace_event_call __used						\
 __attribute__((section("_ftrace_events"))) *__event_##call = &event_##call;
 
+int ftrace_event_is_function(struct ftrace_event_call *call)
+{
+	return call == &event_function;
+}
+
 #include "trace_entries.h"
Signature · 1 line
1.7.1
author
#82 lore raw

PATCH v3 7/8 ftrace, perf: Distinguish ftrace function event field type

In reply to Jiri Olsa

Adding FILTER_TRACE_FN event field type for function tracepoint event, so it can be properly recognized within filtering code.

Currently all fields of ftrace subsystem events share the common field type FILTER_OTHER. Since the function trace fields need special care within the filtering code we need to recognize it properly, hence adding the FILTER_TRACE_FN event type.

---
 include/linux/ftrace_event.h       |    1 +
 kernel/trace/trace_events_filter.c |    7 ++++++-
 kernel/trace/trace_export.c        |   25 ++++++++++++++++++++-----
 3 files changed, 27 insertions(+), 6 deletions(-)
include/linux/ftrace_event.h +1 -0
index 2bf677c..dd478fc 100644
@@ -245,6 +245,7 @@ enum {
 	FILTER_STATIC_STRING,
 	FILTER_DYN_STRING,
 	FILTER_PTR_STRING,
+	FILTER_TRACE_FN,
 };
 
 #define EVENT_STORAGE_SIZE 128
kernel/trace/trace_events_filter.c +6 -1
index f04cc31..66b74ab 100644
@@ -900,6 +900,11 @@ int filter_assign_type(const char *type)
 	return FILTER_OTHER;
 }
 
+static bool is_function_field(struct ftrace_event_field *field)
+{
+	return field->filter_type == FILTER_TRACE_FN;
+}
+
 static bool is_string_field(struct ftrace_event_field *field)
 {
 	return field->filter_type == FILTER_DYN_STRING ||
@@ -987,7 +992,7 @@ static int init_pred(struct filter_parse_state *ps,
 			fn = filter_pred_strloc;
 		else
 			fn = filter_pred_pchar;
-	} else {
+	} else if (!is_function_field(field)) {
 		if (field->is_signed)
 			ret = strict_strtoll(pred->regex.pattern, 0, &val);
 		else
kernel/trace/trace_export.c +20 -5
index 867653c..46c35e2 100644
@@ -67,7 +67,7 @@ static void __always_unused ____ftrace_check_##name(void)	\
 	ret = trace_define_field(event_call, #type, #item,		\
 				 offsetof(typeof(field), item),		\
 				 sizeof(field.item),			\
-				 is_signed_type(type), FILTER_OTHER);	\
+				 is_signed_type(type), filter_type);	\
 	if (ret)							\
 		return ret;
 
@@ -77,7 +77,7 @@ static void __always_unused ____ftrace_check_##name(void)	\
 				 offsetof(typeof(field),		\
 					  container.item),		\
 				 sizeof(field.container.item),		\
-				 is_signed_type(type), FILTER_OTHER);	\
+				 is_signed_type(type), filter_type);	\
 	if (ret)							\
 		return ret;
 
@@ -91,7 +91,7 @@ static void __always_unused ____ftrace_check_##name(void)	\
 		ret = trace_define_field(event_call, event_storage, #item, \
 				 offsetof(typeof(field), item),		\
 				 sizeof(field.item),			\
-				 is_signed_type(type), FILTER_OTHER);	\
+				 is_signed_type(type), filter_type);	\
 		mutex_unlock(&event_storage_mutex);			\
 		if (ret)						\
 			return ret;					\
@@ -104,7 +104,7 @@ static void __always_unused ____ftrace_check_##name(void)	\
 				 offsetof(typeof(field),		\
 					  container.item),		\
 				 sizeof(field.container.item),		\
-				 is_signed_type(type), FILTER_OTHER);	\
+				 is_signed_type(type), filter_type);	\
 	if (ret)							\
 		return ret;
 
@@ -112,10 +112,24 @@ static void __always_unused ____ftrace_check_##name(void)	\
 #define __dynamic_array(type, item)					\
 	ret = trace_define_field(event_call, #type, #item,		\
 				 offsetof(typeof(field), item),		\
-				 0, is_signed_type(type), FILTER_OTHER);\
+				 0, is_signed_type(type), filter_type);\
 	if (ret)							\
 		return ret;
 
+#define FILTER_TYPE_TRACE_FN           FILTER_TRACE_FN
+#define FILTER_TYPE_TRACE_GRAPH_ENT    FILTER_OTHER
+#define FILTER_TYPE_TRACE_GRAPH_RET    FILTER_OTHER
+#define FILTER_TYPE_TRACE_CTX          FILTER_OTHER
+#define FILTER_TYPE_TRACE_WAKE         FILTER_OTHER
+#define FILTER_TYPE_TRACE_STACK                FILTER_OTHER
+#define FILTER_TYPE_TRACE_USER_STACK   FILTER_OTHER
+#define FILTER_TYPE_TRACE_BPRINT       FILTER_OTHER
+#define FILTER_TYPE_TRACE_PRINT                FILTER_OTHER
+#define FILTER_TYPE_TRACE_MMIO_RW      FILTER_OTHER
+#define FILTER_TYPE_TRACE_MMIO_MAP     FILTER_OTHER
+#define FILTER_TYPE_TRACE_BRANCH       FILTER_OTHER
+#define FILTER_TYPE(arg)               FILTER_TYPE_##arg
+
 #undef FTRACE_ENTRY
 #define FTRACE_ENTRY(name, struct_name, id, tstruct, print)		\
 int									\
@@ -123,6 +137,7 @@ ftrace_define_fields_##name(struct ftrace_event_call *event_call)	\
 {									\
 	struct struct_name field;					\
 	int ret;							\
+	int filter_type = FILTER_TYPE(id);				\
 									\
 	tstruct;							\
 									\
Signature · 1 line
1.7.1
author
#83 lore raw

PATCH v3 8/8 ftrace, perf: Add filter support for function trace event

In reply to Jiri Olsa

Adding support to filter function trace event via perf interface. It is now possible to use filter interface in the perf tool like:

  perf record -e ftrace:function --filter="(ip == mm_*)" ls

The filter syntax is restricted to the the 'ip' field only, and following operators are accepted '==' '!=' '||', ending up with the filter strings like:

"ip == f1 f2 ..." || "ip != f3 f4 ..." ...

The '==' operator adds trace filter with same effect as would be added via set_ftrace_filter file.

The '!=' operator adds trace filter with same effect as would be added via set_ftrace_notrace file.

The right side of the '!=', '==' operators is list of functions or regexp. to be added to filter separated by space. Same syntax is supported/required as for the set_ftrace_filter and set_ftrace_notrace files.

The '||' operator is used for connecting multiple filter definitions together. It is possible to have more than one '==' and '!=' operators within one filter string.

---
 kernel/trace/trace.h               |    2 -
 kernel/trace/trace_events_filter.c |  113 +++++++++++++++++++++++++++++++++---
 2 files changed, 105 insertions(+), 10 deletions(-)
kernel/trace/trace.h +0 -2
index e88e58a..4ec6d18 100644
@@ -770,9 +770,7 @@ struct filter_pred {
 	u64 			val;
 	struct regex		regex;
 	unsigned short		*ops;
-#ifdef CONFIG_FTRACE_STARTUP_TEST
 	struct ftrace_event_field *field;
-#endif
 	int 			offset;
 	int 			not;
 	int 			op;
kernel/trace/trace_events_filter.c +105 -8

173 lines not on this page · raw message

Signature · 1 line
1.7.1
author
#84 lore raw

v4 cover ftrace, perf: Adding support to use function trace

In reply to Jiri Olsa

hi,
here's another version of perf support for function trace with filter.

attached patches:
1/7 ftrace: Change filter/notrace set functions to return exit code 2/7 ftrace: Add enable/disable ftrace_ops control interface 3/7 ftrace, perf: Add open/close tracepoint perf registration actions 4/7 ftrace, perf: Add add/del tracepoint perf registration actions 5/7 ftrace, perf: Add support to use function tracepoint in perf 6/7 ftrace, perf: Distinguish ftrace function event field type 7/7 ftrace, perf: Add filter support for function trace event

v4 changes:
2/7 - FL_GLOBAL_CONTROL changed to FL_GLOBAL_CONTROL_MASK
- changed WARN_ON_ONCE() to include the !preempt_count()
- changed this_cpu_ptr to per_cpu_ptr

ommited Fix possible NULL dereferencing in __ftrace_hash_rec_update (2/8 in v3)

v3 changes:
3/8 - renamed __add/remove_ftrace_ops
- fixed preemtp_enable/recursion_clear order in ftrace_ops_control_func
- renamed/commented API functions - enable/disable_ftrace_function

ommited graph tracer workarounf patch 10/10

v2 changes:
01/10 - keeping the old fix instead of adding hash_has_contents func
I'll send separating patchset for this
02/10 - using different way to avoid the issue (3/9 in v1) 03/10 - using the way proposed by Steven for controling ftrace_ops

         (4/9 in v1)
 06/10 - added check ensuring the ftrace:function event could be used by
         root only (7/9 in v1)

08/10 - added more description (8/9 in v1)
09/10 - changed '&&' operator to '||' which seems more suitable
in this case (9/9 in v1)

thanks,
jirka

---
 include/linux/ftrace.h             |   46 ++++++++-
 include/linux/ftrace_event.h       |    9 +-
 include/linux/perf_event.h         |    3 +
 kernel/trace/ftrace.c              |  135 +++++++++++++++++++++--
 kernel/trace/trace.h               |   11 ++-
 kernel/trace/trace_event_perf.c    |  212 +++++++++++++++++++++++++++++-------
 kernel/trace/trace_events.c        |   12 ++-
 kernel/trace/trace_events_filter.c |  116 ++++++++++++++++++-
 kernel/trace/trace_export.c        |   53 ++++++++-
 kernel/trace/trace_kprobe.c        |    8 +-
 kernel/trace/trace_syscalls.c      |   18 +++-
 11 files changed, 542 insertions(+), 81 deletions(-)
author
#85 lore raw

PATCH v5 1/7 ftrace: Change filter/notrace set functions to return exit code

Currently the ftrace_set_filter and ftrace_set_notrace functions do not return any return code. So there's no way for ftrace_ops user to tell wether the filter was correctly applied.

The set_ftrace_filter interface returns error in case the filter did not match:

  # echo krava > set_ftrace_filter
  bash: echo: write error: Invalid argument

Changing both ftrace_set_filter and ftrace_set_notrace functions to return zero if the filter was applied correctly or -E* values in case of error.

---
 include/linux/ftrace.h |    4 ++--
 kernel/trace/ftrace.c  |   16 ++++++++++------
 2 files changed, 12 insertions(+), 8 deletions(-)
include/linux/ftrace.h +2 -2
index 26eafce..523640f 100644
@@ -180,9 +180,9 @@ struct dyn_ftrace {
 };
 
 int ftrace_force_update(void);
-void ftrace_set_filter(struct ftrace_ops *ops, unsigned char *buf,
+int ftrace_set_filter(struct ftrace_ops *ops, unsigned char *buf,
 		       int len, int reset);
-void ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
+int ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
 			int len, int reset);
 void ftrace_set_global_filter(unsigned char *buf, int len, int reset);
 void ftrace_set_global_notrace(unsigned char *buf, int len, int reset);
kernel/trace/ftrace.c +10 -6
index b79ab33..7eb702f 100644
@@ -2912,8 +2912,11 @@ ftrace_set_regex(struct ftrace_ops *ops, unsigned char *buf, int len,
 	mutex_lock(&ftrace_regex_lock);
 	if (reset)
 		ftrace_filter_reset(hash);
-	if (buf)
-		ftrace_match_records(hash, buf, len);
+	if (buf &&
+	    !ftrace_match_records(hash, buf, len)) {
+		ret = -EINVAL;
+		goto out_regex_unlock;
+	}
 
 	mutex_lock(&ftrace_lock);
 	ret = ftrace_hash_move(ops, enable, orig_hash, hash);
@@ -2923,6 +2926,7 @@ ftrace_set_regex(struct ftrace_ops *ops, unsigned char *buf, int len,
 
 	mutex_unlock(&ftrace_lock);
 
+ out_regex_unlock:
 	mutex_unlock(&ftrace_regex_lock);
 
 	free_ftrace_hash(hash);
@@ -2939,10 +2943,10 @@ ftrace_set_regex(struct ftrace_ops *ops, unsigned char *buf, int len,
  * Filters denote which functions should be enabled when tracing is enabled.
  * If @buf is NULL and reset is set, all functions will be enabled for tracing.
  */
-void ftrace_set_filter(struct ftrace_ops *ops, unsigned char *buf,
+int ftrace_set_filter(struct ftrace_ops *ops, unsigned char *buf,
 		       int len, int reset)
 {
-	ftrace_set_regex(ops, buf, len, reset, 1);
+	return ftrace_set_regex(ops, buf, len, reset, 1);
 }
 EXPORT_SYMBOL_GPL(ftrace_set_filter);
 
@@ -2957,10 +2961,10 @@ EXPORT_SYMBOL_GPL(ftrace_set_filter);
  * is enabled. If @buf is NULL and reset is set, all functions will be enabled
  * for tracing.
  */
-void ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
+int ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
 			int len, int reset)
 {
-	ftrace_set_regex(ops, buf, len, reset, 0);
+	return ftrace_set_regex(ops, buf, len, reset, 0);
 }
 EXPORT_SYMBOL_GPL(ftrace_set_notrace);
 /**
Signature · 1 line
1.7.1
maintainer
#86 lore raw

Re: [PATCH 1/7] ftrace: Change filter/notrace set functions to return exit code

kernel/trace/ftrace.c ftrace_set_regex(struct ftrace_ops *ops, unsigned char *buf, int len, · line 2916

Quoted 12 lines · Jiri Olsa
kernel/trace/ftrace.c +2 -2
index b79ab33..7eb702f 100644
@@ -2912,8 +2912,11 @@ ftrace_set_regex(struct ftrace_ops *ops, unsigned char *buf, int len,
 	mutex_lock(&ftrace_regex_lock);
 	if (reset)
 		ftrace_filter_reset(hash);
-	if (buf)
-		ftrace_match_records(hash, buf, len);
+	if (buf &&
+	    !ftrace_match_records(hash, buf, len)) {

I'm fine with this patch, but the above line break is ugly. Put that on one line, and it still is < 80 characters.

I could just take this patch and fix the above. No need to make a new patch set for this one fix.

-- Steve

Quoted 14 lines
kernel/trace/ftrace.c +4 -0
+		ret = -EINVAL;
+		goto out_regex_unlock;
+	}
 
 	mutex_lock(&ftrace_lock);
 	ret = ftrace_hash_move(ops, enable, orig_hash, hash);
@@ -2923,6 +2926,7 @@ ftrace_set_regex(struct ftrace_ops *ops, unsigned char *buf, int len,
 
 	mutex_unlock(&ftrace_lock);
 
+ out_regex_unlock:
 	mutex_unlock(&ftrace_regex_lock);
 
 	free_ftrace_hash(hash);
author
#87 lore raw

PATCH v5 1/7 ftrace: Change filter/notrace set functions to return exit code

Quoted 17 lines · Steven Rostedt

kernel/trace/ftrace.c ftrace_set_regex(struct ftrace_ops *ops, unsigned char *buf, int len, · line 2916

Quoted 12 lines · Jiri Olsa
kernel/trace/ftrace.c +2 -2
index b79ab33..7eb702f 100644
@@ -2912,8 +2912,11 @@ ftrace_set_regex(struct ftrace_ops *ops, unsigned char *buf, int len,
 	mutex_lock(&ftrace_regex_lock);
 	if (reset)
 		ftrace_filter_reset(hash);
-	if (buf)
-		ftrace_match_records(hash, buf, len);
+	if (buf &&
+	    !ftrace_match_records(hash, buf, len)) {

I'm fine with this patch, but the above line break is ugly. Put that on one line, and it still is < 80 characters.

Quoted 2 lines

I could just take this patch and fix the above. No need to make a new patch set for this one fix.

fixed version attached, thanks
jirka

---
Currently the ftrace_set_filter and ftrace_set_notrace functions do not return any return code. So there's no way for ftrace_ops user to tell wether the filter was correctly applied.

The set_ftrace_filter interface returns error in case the filter did not match:

  # echo krava > set_ftrace_filter
  bash: echo: write error: Invalid argument

Changing both ftrace_set_filter and ftrace_set_notrace functions to return zero if the filter was applied correctly or -E* values in case of error.

---
 include/linux/ftrace.h |    4 ++--
 kernel/trace/ftrace.c  |   15 +++++++++------
 2 files changed, 11 insertions(+), 8 deletions(-)
include/linux/ftrace.h +2 -2
index 26eafce..523640f 100644
@@ -180,9 +180,9 @@ struct dyn_ftrace {
 };
 
 int ftrace_force_update(void);
-void ftrace_set_filter(struct ftrace_ops *ops, unsigned char *buf,
+int ftrace_set_filter(struct ftrace_ops *ops, unsigned char *buf,
 		       int len, int reset);
-void ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
+int ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
 			int len, int reset);
 void ftrace_set_global_filter(unsigned char *buf, int len, int reset);
 void ftrace_set_global_notrace(unsigned char *buf, int len, int reset);
kernel/trace/ftrace.c +9 -6
index b79ab33..46e1031 100644
@@ -2912,8 +2912,10 @@ ftrace_set_regex(struct ftrace_ops *ops, unsigned char *buf, int len,
 	mutex_lock(&ftrace_regex_lock);
 	if (reset)
 		ftrace_filter_reset(hash);
-	if (buf)
-		ftrace_match_records(hash, buf, len);
+	if (buf && !ftrace_match_records(hash, buf, len)) {
+		ret = -EINVAL;
+		goto out_regex_unlock;
+	}
 
 	mutex_lock(&ftrace_lock);
 	ret = ftrace_hash_move(ops, enable, orig_hash, hash);
@@ -2923,6 +2925,7 @@ ftrace_set_regex(struct ftrace_ops *ops, unsigned char *buf, int len,
 
 	mutex_unlock(&ftrace_lock);
 
+ out_regex_unlock:
 	mutex_unlock(&ftrace_regex_lock);
 
 	free_ftrace_hash(hash);
@@ -2939,10 +2942,10 @@ ftrace_set_regex(struct ftrace_ops *ops, unsigned char *buf, int len,
  * Filters denote which functions should be enabled when tracing is enabled.
  * If @buf is NULL and reset is set, all functions will be enabled for tracing.
  */
-void ftrace_set_filter(struct ftrace_ops *ops, unsigned char *buf,
+int ftrace_set_filter(struct ftrace_ops *ops, unsigned char *buf,
 		       int len, int reset)
 {
-	ftrace_set_regex(ops, buf, len, reset, 1);
+	return ftrace_set_regex(ops, buf, len, reset, 1);
 }
 EXPORT_SYMBOL_GPL(ftrace_set_filter);
 
@@ -2957,10 +2960,10 @@ EXPORT_SYMBOL_GPL(ftrace_set_filter);
  * is enabled. If @buf is NULL and reset is set, all functions will be enabled
  * for tracing.
  */
-void ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
+int ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
 			int len, int reset)
 {
-	ftrace_set_regex(ops, buf, len, reset, 0);
+	return ftrace_set_regex(ops, buf, len, reset, 0);
 }
 EXPORT_SYMBOL_GPL(ftrace_set_notrace);
 /**
Signature · 1 line
1.7.1
author
#88 lore raw

PATCH v5 2/7 ftrace: Add enable/disable ftrace_ops control interface

In reply to Jiri Olsa

Adding a way to temporarily enable/disable ftrace_ops. The change follows the same way as 'global' ftrace_ops are done.

Introducing 2 global ftrace_ops - control_ops and ftrace_control_list which take over all ftrace_ops registered with FTRACE_OPS_FL_CONTROL flag. In addition new per cpu flag called 'disabled' is also added to ftrace_ops to provide the control information for each cpu.

When ftrace_ops with FTRACE_OPS_FL_CONTROL is registered, it is set as disabled for all cpus.

The ftrace_control_list contains all the registered 'control' ftrace_ops. The control_ops provides function which iterates ftrace_control_list and does the check for 'disabled' flag on current cpu.

Adding 2 inline functions ftrace_function_enable/ftrace_function_disable, which enable/disable the ftrace_ops for given cpu.

---
 include/linux/ftrace.h |   42 +++++++++++++++++
 kernel/trace/ftrace.c  |  119 +++++++++++++++++++++++++++++++++++++++++++++---
 kernel/trace/trace.h   |    2 +
 3 files changed, 156 insertions(+), 7 deletions(-)
include/linux/ftrace.h +42 -0
index 523640f..0d43a2b 100644
@@ -35,12 +35,14 @@ enum {
 	FTRACE_OPS_FL_ENABLED		= 1 << 0,
 	FTRACE_OPS_FL_GLOBAL		= 1 << 1,
 	FTRACE_OPS_FL_DYNAMIC		= 1 << 2,
+	FTRACE_OPS_FL_CONTROL		= 1 << 3,
 };
 
 struct ftrace_ops {
 	ftrace_func_t			func;
 	struct ftrace_ops		*next;
 	unsigned long			flags;
+	void __percpu			*disabled;
 #ifdef CONFIG_DYNAMIC_FTRACE
 	struct ftrace_hash		*notrace_hash;
 	struct ftrace_hash		*filter_hash;
@@ -97,6 +99,46 @@ int register_ftrace_function(struct ftrace_ops *ops);
 int unregister_ftrace_function(struct ftrace_ops *ops);
 void clear_ftrace_function(void);
 
+/**
+ * ftrace_function_enable - enable controlled ftrace_ops on given cpu
+ *
+ * This function enables tracing on given cpu by decreasing
+ * the per cpu control variable.
+ * It must be called with preemption disabled and only on
+ * ftrace_ops registered with FTRACE_OPS_FL_CONTROL.
+ */
+static inline void ftrace_function_enable(struct ftrace_ops *ops, int cpu)
+{
+	atomic_t *disabled;
+
+	if (WARN_ON_ONCE(!(ops->flags & FTRACE_OPS_FL_CONTROL) ||
+			 !preempt_count()))
+		return;
+
+	disabled = per_cpu_ptr(ops->disabled, cpu);
+	atomic_dec(disabled);
+}
+
+/**
+ * ftrace_function_disable - enable controlled ftrace_ops on given cpu
+ *
+ * This function enables tracing on given cpu by decreasing
+ * the per cpu control variable.
+ * It must be called with preemption disabled and only on
+ * ftrace_ops registered with FTRACE_OPS_FL_CONTROL.
+ */
+static inline void ftrace_function_disable(struct ftrace_ops *ops, int cpu)
+{
+	atomic_t *disabled;
+
+	if (WARN_ON_ONCE(!(ops->flags & FTRACE_OPS_FL_CONTROL) ||
+			 !preempt_count()))
+		return;
+
+	disabled = per_cpu_ptr(ops->disabled, cpu);
+	atomic_inc(disabled);
+}
+
 extern void ftrace_stub(unsigned long a0, unsigned long a1);
 
 #else /* !CONFIG_FUNCTION_TRACER */
kernel/trace/ftrace.c +112 -7

174 lines not on this page · raw message

kernel/trace/trace.h +2 -0
index 2c26574..41c54e3 100644
@@ -288,6 +288,8 @@ struct tracer {
 /* for function tracing recursion */
 #define TRACE_INTERNAL_BIT		(1<<11)
 #define TRACE_GLOBAL_BIT		(1<<12)
+#define TRACE_CONTROL_BIT		(1<<13)
+
 /*
  * Abuse of the trace_recursion.
  * As we need a way to maintain state if we are tracing the function
Signature · 1 line
1.7.1
author
#89 lore raw

PATCH v5 3/7 ftrace, perf: Add open/close tracepoint perf registration actions

In reply to Jiri Olsa

Adding TRACE_REG_PERF_OPEN and TRACE_REG_PERF_CLOSE to differentiate register/unregister from open/close actions.

The register/unregister actions are invoked for the first/last tracepoint user when opening/closing the evet.

The open/close actions are invoked for each tracepoint user when opening/closing the event.

---
 include/linux/ftrace_event.h    |    6 +-
 kernel/trace/trace.h            |    5 ++
 kernel/trace/trace_event_perf.c |  116 +++++++++++++++++++++++++--------------
 kernel/trace/trace_events.c     |   10 ++-
 kernel/trace/trace_kprobe.c     |    6 ++-
 kernel/trace/trace_syscalls.c   |   14 +++-
 6 files changed, 106 insertions(+), 51 deletions(-)
include/linux/ftrace_event.h +4 -2
index c3da42d..195e360 100644
@@ -146,6 +146,8 @@ enum trace_reg {
 	TRACE_REG_UNREGISTER,
 	TRACE_REG_PERF_REGISTER,
 	TRACE_REG_PERF_UNREGISTER,
+	TRACE_REG_PERF_OPEN,
+	TRACE_REG_PERF_CLOSE,
 };
 
 struct ftrace_event_call;
@@ -157,7 +159,7 @@ struct ftrace_event_class {
 	void			*perf_probe;
 #endif
 	int			(*reg)(struct ftrace_event_call *event,
-				       enum trace_reg type);
+				       enum trace_reg type, void *data);
 	int			(*define_fields)(struct ftrace_event_call *);
 	struct list_head	*(*get_fields)(struct ftrace_event_call *);
 	struct list_head	fields;
@@ -165,7 +167,7 @@ struct ftrace_event_class {
 };
 
 extern int ftrace_event_reg(struct ftrace_event_call *event,
-			    enum trace_reg type);
+			    enum trace_reg type, void *data);
 
 enum {
 	TRACE_EVENT_FL_ENABLED_BIT,
kernel/trace/trace.h +5 -0
index 41c54e3..85732a8 100644
@@ -828,4 +828,9 @@ extern const char *__stop___trace_bprintk_fmt[];
 	FTRACE_ENTRY(call, struct_name, id, PARAMS(tstruct), PARAMS(print))
 #include "trace_entries.h"
 
+#ifdef CONFIG_PERF_EVENTS
+int perf_ftrace_event_register(struct ftrace_event_call *call,
+			       enum trace_reg type, void *data);
+#endif
+
 #endif /* _LINUX_KERNEL_TRACE_H */
kernel/trace/trace_event_perf.c +75 -41

154 lines not on this page · raw message

kernel/trace/trace_events.c +7 -3
index c212a7f..5138fea 100644
@@ -147,7 +147,8 @@ int trace_event_raw_init(struct ftrace_event_call *call)
 }
 EXPORT_SYMBOL_GPL(trace_event_raw_init);
 
-int ftrace_event_reg(struct ftrace_event_call *call, enum trace_reg type)
+int ftrace_event_reg(struct ftrace_event_call *call,
+		     enum trace_reg type, void *data)
 {
 	switch (type) {
 	case TRACE_REG_REGISTER:
@@ -170,6 +171,9 @@ int ftrace_event_reg(struct ftrace_event_call *call, enum trace_reg type)
 					    call->class->perf_probe,
 					    call);
 		return 0;
+	case TRACE_REG_PERF_OPEN:
+	case TRACE_REG_PERF_CLOSE:
+		return 0;
 #endif
 	}
 	return 0;
@@ -209,7 +213,7 @@ static int ftrace_event_enable_disable(struct ftrace_event_call *call,
 				tracing_stop_cmdline_record();
 				call->flags &= ~TRACE_EVENT_FL_RECORDED_CMD;
 			}
-			call->class->reg(call, TRACE_REG_UNREGISTER);
+			call->class->reg(call, TRACE_REG_UNREGISTER, NULL);
 		}
 		break;
 	case 1:
@@ -218,7 +222,7 @@ static int ftrace_event_enable_disable(struct ftrace_event_call *call,
 				tracing_start_cmdline_record();
 				call->flags |= TRACE_EVENT_FL_RECORDED_CMD;
 			}
-			ret = call->class->reg(call, TRACE_REG_REGISTER);
+			ret = call->class->reg(call, TRACE_REG_REGISTER, NULL);
 			if (ret) {
 				tracing_stop_cmdline_record();
 				pr_info("event trace: Could not enable event "
kernel/trace/trace_kprobe.c +5 -1
index 00d527c..5667f89 100644
@@ -1892,7 +1892,8 @@ static __kprobes void kretprobe_perf_func(struct kretprobe_instance *ri,
 #endif	/* CONFIG_PERF_EVENTS */
 
 static __kprobes
-int kprobe_register(struct ftrace_event_call *event, enum trace_reg type)
+int kprobe_register(struct ftrace_event_call *event,
+		    enum trace_reg type, void *data)
 {
 	struct trace_probe *tp = (struct trace_probe *)event->data;
 
@@ -1909,6 +1910,9 @@ int kprobe_register(struct ftrace_event_call *event, enum trace_reg type)
 	case TRACE_REG_PERF_UNREGISTER:
 		disable_trace_probe(tp, TP_FLAG_PROFILE);
 		return 0;
+	case TRACE_REG_PERF_OPEN:
+	case TRACE_REG_PERF_CLOSE:
+		return 0;
 #endif
 	}
 	return 0;
kernel/trace/trace_syscalls.c +10 -4
index 5f35f6f..8599c1d 100644
@@ -18,9 +18,9 @@ static DECLARE_BITMAP(enabled_enter_syscalls, NR_syscalls);
 static DECLARE_BITMAP(enabled_exit_syscalls, NR_syscalls);
 
 static int syscall_enter_register(struct ftrace_event_call *event,
-				 enum trace_reg type);
+				 enum trace_reg type, void *data);
 static int syscall_exit_register(struct ftrace_event_call *event,
-				 enum trace_reg type);
+				 enum trace_reg type, void *data);
 
 static int syscall_enter_define_fields(struct ftrace_event_call *call);
 static int syscall_exit_define_fields(struct ftrace_event_call *call);
@@ -650,7 +650,7 @@ void perf_sysexit_disable(struct ftrace_event_call *call)
 #endif /* CONFIG_PERF_EVENTS */
 
 static int syscall_enter_register(struct ftrace_event_call *event,
-				 enum trace_reg type)
+				 enum trace_reg type, void *data)
 {
 	switch (type) {
 	case TRACE_REG_REGISTER:
@@ -665,13 +665,16 @@ static int syscall_enter_register(struct ftrace_event_call *event,
 	case TRACE_REG_PERF_UNREGISTER:
 		perf_sysenter_disable(event);
 		return 0;
+	case TRACE_REG_PERF_OPEN:
+	case TRACE_REG_PERF_CLOSE:
+		return 0;
 #endif
 	}
 	return 0;
 }
 
 static int syscall_exit_register(struct ftrace_event_call *event,
-				 enum trace_reg type)
+				 enum trace_reg type, void *data)
 {
 	switch (type) {
 	case TRACE_REG_REGISTER:
@@ -686,6 +689,9 @@ static int syscall_exit_register(struct ftrace_event_call *event,
 	case TRACE_REG_PERF_UNREGISTER:
 		perf_sysexit_disable(event);
 		return 0;
+	case TRACE_REG_PERF_OPEN:
+	case TRACE_REG_PERF_CLOSE:
+		return 0;
 #endif
 	}
 	return 0;
Signature · 1 line
1.7.1
author
#90 lore raw

PATCH v5 4/7 ftrace, perf: Add add/del tracepoint perf registration actions

In reply to Jiri Olsa

Adding TRACE_REG_PERF_ADD and TRACE_REG_PERF_DEL to handle perf event schedule in/out actions.

The add action is invoked for when the perf event is scheduled in, while the del action is invoked when the event is scheduled out.

---
 include/linux/ftrace_event.h    |    2 ++
 kernel/trace/trace_event_perf.c |    4 +++-
 kernel/trace/trace_events.c     |    2 ++
 kernel/trace/trace_kprobe.c     |    2 ++
 kernel/trace/trace_syscalls.c   |    4 ++++
 5 files changed, 13 insertions(+), 1 deletions(-)
include/linux/ftrace_event.h +2 -0
index 195e360..2bf677c 100644
@@ -148,6 +148,8 @@ enum trace_reg {
 	TRACE_REG_PERF_UNREGISTER,
 	TRACE_REG_PERF_OPEN,
 	TRACE_REG_PERF_CLOSE,
+	TRACE_REG_PERF_ADD,
+	TRACE_REG_PERF_DEL,
 };
 
 struct ftrace_event_call;
kernel/trace/trace_event_perf.c +3 -1
index 0cfcc37..d72af0b 100644
@@ -211,12 +211,14 @@ int perf_trace_add(struct perf_event *p_event, int flags)
 	list = this_cpu_ptr(pcpu_list);
 	hlist_add_head_rcu(&p_event->hlist_entry, list);
 
-	return 0;
+	return tp_event->class->reg(tp_event, TRACE_REG_PERF_ADD, p_event);
 }
 
 void perf_trace_del(struct perf_event *p_event, int flags)
 {
+	struct ftrace_event_call *tp_event = p_event->tp_event;
 	hlist_del_rcu(&p_event->hlist_entry);
+	tp_event->class->reg(tp_event, TRACE_REG_PERF_DEL, p_event);
 }
 
 __kprobes void *perf_trace_buf_prepare(int size, unsigned short type,
kernel/trace/trace_events.c +2 -0
index 5138fea..079a93a 100644
@@ -173,6 +173,8 @@ int ftrace_event_reg(struct ftrace_event_call *call,
 		return 0;
 	case TRACE_REG_PERF_OPEN:
 	case TRACE_REG_PERF_CLOSE:
+	case TRACE_REG_PERF_ADD:
+	case TRACE_REG_PERF_DEL:
 		return 0;
 #endif
 	}
kernel/trace/trace_kprobe.c +2 -0
index 5667f89..580a05e 100644
@@ -1912,6 +1912,8 @@ int kprobe_register(struct ftrace_event_call *event,
 		return 0;
 	case TRACE_REG_PERF_OPEN:
 	case TRACE_REG_PERF_CLOSE:
+	case TRACE_REG_PERF_ADD:
+	case TRACE_REG_PERF_DEL:
 		return 0;
 #endif
 	}
kernel/trace/trace_syscalls.c +4 -0
index 8599c1d..5e4f62e 100644
@@ -667,6 +667,8 @@ static int syscall_enter_register(struct ftrace_event_call *event,
 		return 0;
 	case TRACE_REG_PERF_OPEN:
 	case TRACE_REG_PERF_CLOSE:
+	case TRACE_REG_PERF_ADD:
+	case TRACE_REG_PERF_DEL:
 		return 0;
 #endif
 	}
@@ -691,6 +693,8 @@ static int syscall_exit_register(struct ftrace_event_call *event,
 		return 0;
 	case TRACE_REG_PERF_OPEN:
 	case TRACE_REG_PERF_CLOSE:
+	case TRACE_REG_PERF_ADD:
+	case TRACE_REG_PERF_DEL:
 		return 0;
 #endif
 	}
Signature · 1 line
1.7.1
author
#91 lore raw

PATCH v5 5/7 ftrace, perf: Add support to use function tracepoint in perf

In reply to Jiri Olsa

Adding perf registration support for the ftrace function event, so it is now possible to register it via perf interface.

The perf_event struct statically contains ftrace_ops as a handle for function tracer. The function tracer is registered/unregistered in open/close actions, and enabled/disabled in add/del actions.

It is now possible to use function trace within perf commands like:

perf record -e ftrace:function ls
perf stat -e ftrace:function ls

Allowed only for root.

---
 include/linux/perf_event.h      |    3 +
 kernel/trace/trace.h            |    2 +
 kernel/trace/trace_event_perf.c |   92 +++++++++++++++++++++++++++++++++++++++
 kernel/trace/trace_export.c     |   28 ++++++++++++
 4 files changed, 125 insertions(+), 0 deletions(-)
include/linux/perf_event.h +3 -0
index 0b91db2..5003be6 100644
@@ -858,6 +858,9 @@ struct perf_event {
 #ifdef CONFIG_EVENT_TRACING
 	struct ftrace_event_call	*tp_event;
 	struct event_filter		*filter;
+#ifdef CONFIG_FUNCTION_TRACER
+	struct ftrace_ops               ftrace_ops;
+#endif
 #endif
 
 #ifdef CONFIG_CGROUP_PERF
kernel/trace/trace.h +2 -0
index 85732a8..e88e58a 100644
@@ -591,6 +591,8 @@ static inline int ftrace_trace_task(struct task_struct *task)
 static inline int ftrace_is_dead(void) { return 0; }
 #endif
 
+int ftrace_event_is_function(struct ftrace_event_call *call);
+
 /*
  * struct trace_parser - servers for reading the user input separated by spaces
  * @cont: set if the input is not complete - no final space char was found
kernel/trace/trace_event_perf.c +92 -0

101 lines not on this page · raw message

kernel/trace/trace_export.c +28 -0
index bbeec31..867653c 100644
@@ -131,6 +131,28 @@ ftrace_define_fields_##name(struct ftrace_event_call *event_call)	\
 
 #include "trace_entries.h"
 
+static int ftrace_event_class_register(struct ftrace_event_call *call,
+				       enum trace_reg type, void *data)
+{
+	switch (type) {
+	case TRACE_REG_PERF_REGISTER:
+	case TRACE_REG_PERF_UNREGISTER:
+		return 0;
+	case TRACE_REG_PERF_OPEN:
+	case TRACE_REG_PERF_CLOSE:
+	case TRACE_REG_PERF_ADD:
+	case TRACE_REG_PERF_DEL:
+#ifdef CONFIG_PERF_EVENTS
+		return perf_ftrace_event_register(call, type, data);
+#endif
+	case TRACE_REG_REGISTER:
+	case TRACE_REG_UNREGISTER:
+		break;
+	}
+
+	return -EINVAL;
+}
+
 #undef __entry
 #define __entry REC
 
@@ -159,6 +181,7 @@ struct ftrace_event_class event_class_ftrace_##call = {			\
 	.system			= __stringify(TRACE_SYSTEM),		\
 	.define_fields		= ftrace_define_fields_##call,		\
 	.fields			= LIST_HEAD_INIT(event_class_ftrace_##call.fields),\
+	.reg			= ftrace_event_class_register,		\
 };									\
 									\
 struct ftrace_event_call __used event_##call = {			\
@@ -170,4 +193,9 @@ struct ftrace_event_call __used event_##call = {			\
 struct ftrace_event_call __used						\
 __attribute__((section("_ftrace_events"))) *__event_##call = &event_##call;
 
+int ftrace_event_is_function(struct ftrace_event_call *call)
+{
+	return call == &event_function;
+}
+
 #include "trace_entries.h"
Signature · 1 line
1.7.1
author
#92 lore raw

PATCH v5 6/7 ftrace, perf: Distinguish ftrace function event field type

In reply to Jiri Olsa

Adding FILTER_TRACE_FN event field type for function tracepoint event, so it can be properly recognized within filtering code.

Currently all fields of ftrace subsystem events share the common field type FILTER_OTHER. Since the function trace fields need special care within the filtering code we need to recognize it properly, hence adding the FILTER_TRACE_FN event type.

---
 include/linux/ftrace_event.h       |    1 +
 kernel/trace/trace_events_filter.c |    7 ++++++-
 kernel/trace/trace_export.c        |   25 ++++++++++++++++++++-----
 3 files changed, 27 insertions(+), 6 deletions(-)
include/linux/ftrace_event.h +1 -0
index 2bf677c..dd478fc 100644
@@ -245,6 +245,7 @@ enum {
 	FILTER_STATIC_STRING,
 	FILTER_DYN_STRING,
 	FILTER_PTR_STRING,
+	FILTER_TRACE_FN,
 };
 
 #define EVENT_STORAGE_SIZE 128
kernel/trace/trace_events_filter.c +6 -1
index f04cc31..66b74ab 100644
@@ -900,6 +900,11 @@ int filter_assign_type(const char *type)
 	return FILTER_OTHER;
 }
 
+static bool is_function_field(struct ftrace_event_field *field)
+{
+	return field->filter_type == FILTER_TRACE_FN;
+}
+
 static bool is_string_field(struct ftrace_event_field *field)
 {
 	return field->filter_type == FILTER_DYN_STRING ||
@@ -987,7 +992,7 @@ static int init_pred(struct filter_parse_state *ps,
 			fn = filter_pred_strloc;
 		else
 			fn = filter_pred_pchar;
-	} else {
+	} else if (!is_function_field(field)) {
 		if (field->is_signed)
 			ret = strict_strtoll(pred->regex.pattern, 0, &val);
 		else
kernel/trace/trace_export.c +20 -5
index 867653c..46c35e2 100644
@@ -67,7 +67,7 @@ static void __always_unused ____ftrace_check_##name(void)	\
 	ret = trace_define_field(event_call, #type, #item,		\
 				 offsetof(typeof(field), item),		\
 				 sizeof(field.item),			\
-				 is_signed_type(type), FILTER_OTHER);	\
+				 is_signed_type(type), filter_type);	\
 	if (ret)							\
 		return ret;
 
@@ -77,7 +77,7 @@ static void __always_unused ____ftrace_check_##name(void)	\
 				 offsetof(typeof(field),		\
 					  container.item),		\
 				 sizeof(field.container.item),		\
-				 is_signed_type(type), FILTER_OTHER);	\
+				 is_signed_type(type), filter_type);	\
 	if (ret)							\
 		return ret;
 
@@ -91,7 +91,7 @@ static void __always_unused ____ftrace_check_##name(void)	\
 		ret = trace_define_field(event_call, event_storage, #item, \
 				 offsetof(typeof(field), item),		\
 				 sizeof(field.item),			\
-				 is_signed_type(type), FILTER_OTHER);	\
+				 is_signed_type(type), filter_type);	\
 		mutex_unlock(&event_storage_mutex);			\
 		if (ret)						\
 			return ret;					\
@@ -104,7 +104,7 @@ static void __always_unused ____ftrace_check_##name(void)	\
 				 offsetof(typeof(field),		\
 					  container.item),		\
 				 sizeof(field.container.item),		\
-				 is_signed_type(type), FILTER_OTHER);	\
+				 is_signed_type(type), filter_type);	\
 	if (ret)							\
 		return ret;
 
@@ -112,10 +112,24 @@ static void __always_unused ____ftrace_check_##name(void)	\
 #define __dynamic_array(type, item)					\
 	ret = trace_define_field(event_call, #type, #item,		\
 				 offsetof(typeof(field), item),		\
-				 0, is_signed_type(type), FILTER_OTHER);\
+				 0, is_signed_type(type), filter_type);\
 	if (ret)							\
 		return ret;
 
+#define FILTER_TYPE_TRACE_FN           FILTER_TRACE_FN
+#define FILTER_TYPE_TRACE_GRAPH_ENT    FILTER_OTHER
+#define FILTER_TYPE_TRACE_GRAPH_RET    FILTER_OTHER
+#define FILTER_TYPE_TRACE_CTX          FILTER_OTHER
+#define FILTER_TYPE_TRACE_WAKE         FILTER_OTHER
+#define FILTER_TYPE_TRACE_STACK                FILTER_OTHER
+#define FILTER_TYPE_TRACE_USER_STACK   FILTER_OTHER
+#define FILTER_TYPE_TRACE_BPRINT       FILTER_OTHER
+#define FILTER_TYPE_TRACE_PRINT                FILTER_OTHER
+#define FILTER_TYPE_TRACE_MMIO_RW      FILTER_OTHER
+#define FILTER_TYPE_TRACE_MMIO_MAP     FILTER_OTHER
+#define FILTER_TYPE_TRACE_BRANCH       FILTER_OTHER
+#define FILTER_TYPE(arg)               FILTER_TYPE_##arg
+
 #undef FTRACE_ENTRY
 #define FTRACE_ENTRY(name, struct_name, id, tstruct, print)		\
 int									\
@@ -123,6 +137,7 @@ ftrace_define_fields_##name(struct ftrace_event_call *event_call)	\
 {									\
 	struct struct_name field;					\
 	int ret;							\
+	int filter_type = FILTER_TYPE(id);				\
 									\
 	tstruct;							\
 									\
Signature · 1 line
1.7.1
author
#93 lore raw

PATCH v5 7/7 ftrace, perf: Add filter support for function trace event

In reply to Jiri Olsa

Adding support to filter function trace event via perf interface. It is now possible to use filter interface in the perf tool like:

  perf record -e ftrace:function --filter="(ip == mm_*)" ls

The filter syntax is restricted to the the 'ip' field only, and following operators are accepted '==' '!=' '||', ending up with the filter strings like:

"ip == f1 f2 ..." || "ip != f3 f4 ..." ...

The '==' operator adds trace filter with same effect as would be added via set_ftrace_filter file.

The '!=' operator adds trace filter with same effect as would be added via set_ftrace_notrace file.

The right side of the '!=', '==' operators is list of functions or regexp. to be added to filter separated by space. Same syntax is supported/required as for the set_ftrace_filter and set_ftrace_notrace files.

The '||' operator is used for connecting multiple filter definitions together. It is possible to have more than one '==' and '!=' operators within one filter string.

---
 kernel/trace/trace.h               |    2 -
 kernel/trace/trace_events_filter.c |  113 +++++++++++++++++++++++++++++++++---
 2 files changed, 105 insertions(+), 10 deletions(-)
kernel/trace/trace.h +0 -2
index e88e58a..4ec6d18 100644
@@ -770,9 +770,7 @@ struct filter_pred {
 	u64 			val;
 	struct regex		regex;
 	unsigned short		*ops;
-#ifdef CONFIG_FTRACE_STARTUP_TEST
 	struct ftrace_event_field *field;
-#endif
 	int 			offset;
 	int 			not;
 	int 			op;
kernel/trace/trace_events_filter.c +105 -8

173 lines not on this page · raw message

Signature · 1 line
1.7.1
#94 lore raw

Re: [PATCH 7/7] ftrace, perf: Add filter support for function trace event

Quoted 11 lines · Jiri Olsa

Adding support to filter function trace event via perf interface. It is now possible to use filter interface in the perf tool like:

  perf record -e ftrace:function --filter="(ip == mm_*)" ls

The filter syntax is restricted to the the 'ip' field only, and following operators are accepted '==' '!=' '||', ending up with the filter strings like:

"ip == f1 f2 ..." || "ip != f3 f4 ..." ...

Having the functions seperated like this sort of violates the grammar of the filtering interface.

The typical way to do this would have been to stringify the
functions: ip == "f1 f2"

I feel a bit uncomfortable with "ip == f1 f2" scheme but perhaps we can live with that. Especially as otherwise that would require us to type "ip == \"f1 f2\"" for the whole filtering expression.

Thoughts?

author
#95 lore raw

Re: [PATCH 7/7] ftrace, perf: Add filter support for function trace event

Quoted 16 lines · Frederic Weisbecker
Quoted 11 lines · Jiri Olsa

Adding support to filter function trace event via perf interface. It is now possible to use filter interface in the perf tool like:

  perf record -e ftrace:function --filter="(ip == mm_*)" ls

The filter syntax is restricted to the the 'ip' field only, and following operators are accepted '==' '!=' '||', ending up with the filter strings like:

"ip == f1 f2 ..." || "ip != f3 f4 ..." ...

Having the functions seperated like this sort of violates the grammar of the filtering interface.

Quoted 6 lines

The typical way to do this would have been to stringify the
functions: ip == "f1 f2"

I feel a bit uncomfortable with "ip == f1 f2" scheme but perhaps we can live with that. Especially as otherwise that would require us to type "ip == \"f1 f2\"" for the whole filtering expression.

ugh, just realized there's a problem with this in the patch actually, and it's not working as expected. I'll send out new version soon..

thanks,
jirka

Quoted 2 lines

Thoughts?

author
#96 lore raw

Re: [PATCHvFIXED 7/7] ftrace, perf: Add filter support for function trace event

Quoted 24 lines · Jiri Olsa
Quoted 16 lines · Frederic Weisbecker
Quoted 11 lines · Jiri Olsa

Adding support to filter function trace event via perf interface. It is now possible to use filter interface in the perf tool like:

  perf record -e ftrace:function --filter="(ip == mm_*)" ls

The filter syntax is restricted to the the 'ip' field only, and following operators are accepted '==' '!=' '||', ending up with the filter strings like:

"ip == f1 f2 ..." || "ip != f3 f4 ..." ...

Having the functions seperated like this sort of violates the grammar of the filtering interface.

Quoted 6 lines

The typical way to do this would have been to stringify the
functions: ip == "f1 f2"

I feel a bit uncomfortable with "ip == f1 f2" scheme but perhaps we can live with that. Especially as otherwise that would require us to type "ip == \"f1 f2\"" for the whole filtering expression.

Quoted 8 lines

ugh, just realized there's a problem with this in the patch actually, and it's not working as expected. I'll send out new version soon..

thanks,
jirka

Quoted 2 lines

Thoughts?

how about this one.. ;)

for some reason I presumed ftrace_set_filter would deal with ' ' as a filter separator.. but it needs to be done before we use this function... (ftrace_set_notrace respectively)

so now you could use one of following:

perf record -e ftrace:function --filter '(ip == do_execve,sys_*,ext*)' ls perf record -e ftrace:function --filter '(ip == "do_execve,sys_*,ext*")' ls perf record -e ftrace:function --filter '(ip == "do_execve sys_* ext*")' ls

jirka

---
Adding support to filter function trace event via perf interface. It is now possible to use filter interface in the perf tool like:

  perf record -e ftrace:function --filter="(ip == mm_*)" ls

The filter syntax is restricted to the the 'ip' field only, and following operators are accepted '==' '!=' '||', ending up with the filter strings like:

ip == f1[, ]f2 ... || ip != f3[, ]f4 ...

with comma ',' or space ' ' as a function separator. If the space ' ' is used as a separator, the right side of the assignment needs to be enclosed in double quotes '"'.

The '==' operator adds trace filter with same effect as would be added via set_ftrace_filter file.

The '!=' operator adds trace filter with same effect as would be added via set_ftrace_notrace file.

The right side of the '!=', '==' operators is list of functions or regexp. to be added to filter separated by space.

The '||' operator is used for connecting multiple filter definitions together. It is possible to have more than one '==' and '!=' operators within one filter string.

---
 include/linux/ftrace.h             |    1 +
 kernel/trace/ftrace.c              |    6 ++
 kernel/trace/trace.h               |    2 -
 kernel/trace/trace_event_perf.c    |    4 +-
 kernel/trace/trace_events_filter.c |  162 ++++++++++++++++++++++++++++++++++--
 5 files changed, 164 insertions(+), 11 deletions(-)
include/linux/ftrace.h +1 -0
index 0d43a2b..40bf05f 100644
@@ -228,6 +228,7 @@ int ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
 			int len, int reset);
 void ftrace_set_global_filter(unsigned char *buf, int len, int reset);
 void ftrace_set_global_notrace(unsigned char *buf, int len, int reset);
+void ftrace_free_filter(struct ftrace_ops *ops);
 
 int register_ftrace_command(struct ftrace_func_command *cmd);
 int unregister_ftrace_command(struct ftrace_func_command *cmd);
kernel/trace/ftrace.c +6 -0
index 7af5fb3..693df34 100644
@@ -1193,6 +1193,12 @@ static void free_ftrace_hash_rcu(struct ftrace_hash *hash)
 	call_rcu_sched(&hash->rcu, __free_ftrace_hash_rcu);
 }
 
+void ftrace_free_filter(struct ftrace_ops *ops)
+{
+	free_ftrace_hash(ops->filter_hash);
+	free_ftrace_hash(ops->notrace_hash);
+}
+
 static struct ftrace_hash *alloc_ftrace_hash(int size_bits)
 {
 	struct ftrace_hash *hash;
kernel/trace/trace.h +0 -2
index e88e58a..4ec6d18 100644
@@ -770,9 +770,7 @@ struct filter_pred {
 	u64 			val;
 	struct regex		regex;
 	unsigned short		*ops;
-#ifdef CONFIG_FTRACE_STARTUP_TEST
 	struct ftrace_event_field *field;
-#endif
 	int 			offset;
 	int 			not;
 	int 			op;
kernel/trace/trace_event_perf.c +3 -1
index 57eb232..220b50a 100644
@@ -298,7 +298,9 @@ static int perf_ftrace_function_register(struct perf_event *event)
 static int perf_ftrace_function_unregister(struct perf_event *event)
 {
 	struct ftrace_ops *ops = &event->ftrace_ops;
-	return unregister_ftrace_function(ops);
+	int ret = unregister_ftrace_function(ops);
+	ftrace_free_filter(ops);
+	return ret;
 }
 
 static void perf_ftrace_function_enable(struct perf_event *event)
kernel/trace/trace_events_filter.c +154 -8

222 lines not on this page · raw message

Signature · 1 line
1.7.1
#97 lore raw

Re: [PATCHvFIXED 7/7] ftrace, perf: Add filter support for function trace event

Quoted 40 lines · Jiri Olsa
Quoted 24 lines · Jiri Olsa
Quoted 16 lines · Frederic Weisbecker
Quoted 11 lines · Jiri Olsa

Adding support to filter function trace event via perf interface. It is now possible to use filter interface in the perf tool like:

  perf record -e ftrace:function --filter="(ip == mm_*)" ls

The filter syntax is restricted to the the 'ip' field only, and following operators are accepted '==' '!=' '||', ending up with the filter strings like:

"ip == f1 f2 ..." || "ip != f3 f4 ..." ...

Having the functions seperated like this sort of violates the grammar of the filtering interface.

Quoted 6 lines

The typical way to do this would have been to stringify the
functions: ip == "f1 f2"

I feel a bit uncomfortable with "ip == f1 f2" scheme but perhaps we can live with that. Especially as otherwise that would require us to type "ip == \"f1 f2\"" for the whole filtering expression.

Quoted 8 lines

ugh, just realized there's a problem with this in the patch actually, and it's not working as expected. I'll send out new version soon..

thanks,
jirka

Quoted 2 lines

Thoughts?

how about this one.. ;)

for some reason I presumed ftrace_set_filter would deal with ' ' as a filter separator.. but it needs to be done before we use this function... (ftrace_set_notrace respectively)

Quoted 5 lines

so now you could use one of following:

perf record -e ftrace:function --filter '(ip == do_execve,sys_*,ext*)' ls perf record -e ftrace:function --filter '(ip == "do_execve,sys_*,ext*")' ls perf record -e ftrace:function --filter '(ip == "do_execve sys_* ext*")' ls

Nice! ;)

Thanks!

author
#98 lore raw
In reply to Jiri Olsa

should be 0/7 instead of 0/8 ;)

Quoted 55 lines · Jiri Olsa

hi,
here's another version of perf support for function trace with filter.

attached patches:
1/7 ftrace: Change filter/notrace set functions to return exit code 2/7 ftrace: Add enable/disable ftrace_ops control interface 3/7 ftrace, perf: Add open/close tracepoint perf registration actions 4/7 ftrace, perf: Add add/del tracepoint perf registration actions 5/7 ftrace, perf: Add support to use function tracepoint in perf 6/7 ftrace, perf: Distinguish ftrace function event field type 7/7 ftrace, perf: Add filter support for function trace event

v4 changes:
2/7 - FL_GLOBAL_CONTROL changed to FL_GLOBAL_CONTROL_MASK
- changed WARN_ON_ONCE() to include the !preempt_count()
- changed this_cpu_ptr to per_cpu_ptr

ommited Fix possible NULL dereferencing in __ftrace_hash_rec_update (2/8 in v3)

v3 changes:
3/8 - renamed __add/remove_ftrace_ops
- fixed preemtp_enable/recursion_clear order in ftrace_ops_control_func
- renamed/commented API functions - enable/disable_ftrace_function

ommited graph tracer workarounf patch 10/10

v2 changes:
01/10 - keeping the old fix instead of adding hash_has_contents func
I'll send separating patchset for this
02/10 - using different way to avoid the issue (3/9 in v1) 03/10 - using the way proposed by Steven for controling ftrace_ops

         (4/9 in v1)
 06/10 - added check ensuring the ftrace:function event could be used by
         root only (7/9 in v1)

08/10 - added more description (8/9 in v1)
09/10 - changed '&&' operator to '||' which seems more suitable
in this case (9/9 in v1)

thanks,
jirka

---
 include/linux/ftrace.h             |   46 ++++++++-
 include/linux/ftrace_event.h       |    9 +-
 include/linux/perf_event.h         |    3 +
 kernel/trace/ftrace.c              |  135 +++++++++++++++++++++--
 kernel/trace/trace.h               |   11 ++-
 kernel/trace/trace_event_perf.c    |  212 +++++++++++++++++++++++++++++-------
 kernel/trace/trace_events.c        |   12 ++-
 kernel/trace/trace_events_filter.c |  116 ++++++++++++++++++-
 kernel/trace/trace_export.c        |   53 ++++++++-
 kernel/trace/trace_kprobe.c        |    8 +-
 kernel/trace/trace_syscalls.c      |   18 +++-
 11 files changed, 542 insertions(+), 81 deletions(-)
author
#99 lore raw

v5 cover ftrace, perf: Adding support to use function trace

In reply to Jiri Olsa

hi,
here's another version of perf support for function trace with filter.

attached patches:
1/7 ftrace: Change filter/notrace set functions to return exit code 2/7 ftrace: Add enable/disable ftrace_ops control interface 3/7 ftrace, perf: Add open/close tracepoint perf registration actions 4/7 ftrace, perf: Add add/del tracepoint perf registration actions 5/7 ftrace, perf: Add support to use function tracepoint in perf 6/7 ftrace, perf: Distinguish ftrace function event field type 7/7 ftrace, perf: Add filter support for function trace event

v5 changes:
7/7 - fixed to properly support ',' in filter expressions

v4 changes:
2/7 - FL_GLOBAL_CONTROL changed to FL_GLOBAL_CONTROL_MASK
- changed WARN_ON_ONCE() to include the !preempt_count()
- changed this_cpu_ptr to per_cpu_ptr

ommited Fix possible NULL dereferencing in __ftrace_hash_rec_update (2/8 in v3)

v3 changes:
3/8 - renamed __add/remove_ftrace_ops
- fixed preemtp_enable/recursion_clear order in ftrace_ops_control_func
- renamed/commented API functions - enable/disable_ftrace_function

ommited graph tracer workarounf patch 10/10

v2 changes:
01/10 - keeping the old fix instead of adding hash_has_contents func
I'll send separating patchset for this
02/10 - using different way to avoid the issue (3/9 in v1) 03/10 - using the way proposed by Steven for controling ftrace_ops

         (4/9 in v1)
 06/10 - added check ensuring the ftrace:function event could be used by
         root only (7/9 in v1)

08/10 - added more description (8/9 in v1)
09/10 - changed '&&' operator to '||' which seems more suitable
in this case (9/9 in v1)

thanks,
jirka

---
 include/linux/ftrace.h             |   47 ++++++++-
 include/linux/ftrace_event.h       |    9 +-
 include/linux/perf_event.h         |    3 +
 kernel/trace/ftrace.c              |  140 +++++++++++++++++++++--
 kernel/trace/trace.h               |   11 ++-
 kernel/trace/trace_event_perf.c    |  214 +++++++++++++++++++++++++++++-------
 kernel/trace/trace_events.c        |   12 ++-
 kernel/trace/trace_events_filter.c |  164 ++++++++++++++++++++++++++-
 kernel/trace/trace_export.c        |   53 ++++++++-
 kernel/trace/trace_kprobe.c        |    8 +-
 kernel/trace/trace_syscalls.c      |   18 +++-
 11 files changed, 598 insertions(+), 81 deletions(-)
author
#100 lore raw

PATCH v5 1/7 ftrace: Change filter/notrace set functions to return exit code

Currently the ftrace_set_filter and ftrace_set_notrace functions do not return any return code. So there's no way for ftrace_ops user to tell wether the filter was correctly applied.

The set_ftrace_filter interface returns error in case the filter did not match:

  # echo krava > set_ftrace_filter
  bash: echo: write error: Invalid argument

Changing both ftrace_set_filter and ftrace_set_notrace functions to return zero if the filter was applied correctly or -E* values in case of error.

---
 include/linux/ftrace.h |    4 ++--
 kernel/trace/ftrace.c  |   15 +++++++++------
 2 files changed, 11 insertions(+), 8 deletions(-)
include/linux/ftrace.h +2 -2
index 26eafce..523640f 100644
@@ -180,9 +180,9 @@ struct dyn_ftrace {
 };
 
 int ftrace_force_update(void);
-void ftrace_set_filter(struct ftrace_ops *ops, unsigned char *buf,
+int ftrace_set_filter(struct ftrace_ops *ops, unsigned char *buf,
 		       int len, int reset);
-void ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
+int ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
 			int len, int reset);
 void ftrace_set_global_filter(unsigned char *buf, int len, int reset);
 void ftrace_set_global_notrace(unsigned char *buf, int len, int reset);
kernel/trace/ftrace.c +9 -6
index b79ab33..46e1031 100644
@@ -2912,8 +2912,10 @@ ftrace_set_regex(struct ftrace_ops *ops, unsigned char *buf, int len,
 	mutex_lock(&ftrace_regex_lock);
 	if (reset)
 		ftrace_filter_reset(hash);
-	if (buf)
-		ftrace_match_records(hash, buf, len);
+	if (buf && !ftrace_match_records(hash, buf, len)) {
+		ret = -EINVAL;
+		goto out_regex_unlock;
+	}
 
 	mutex_lock(&ftrace_lock);
 	ret = ftrace_hash_move(ops, enable, orig_hash, hash);
@@ -2923,6 +2925,7 @@ ftrace_set_regex(struct ftrace_ops *ops, unsigned char *buf, int len,
 
 	mutex_unlock(&ftrace_lock);
 
+ out_regex_unlock:
 	mutex_unlock(&ftrace_regex_lock);
 
 	free_ftrace_hash(hash);
@@ -2939,10 +2942,10 @@ ftrace_set_regex(struct ftrace_ops *ops, unsigned char *buf, int len,
  * Filters denote which functions should be enabled when tracing is enabled.
  * If @buf is NULL and reset is set, all functions will be enabled for tracing.
  */
-void ftrace_set_filter(struct ftrace_ops *ops, unsigned char *buf,
+int ftrace_set_filter(struct ftrace_ops *ops, unsigned char *buf,
 		       int len, int reset)
 {
-	ftrace_set_regex(ops, buf, len, reset, 1);
+	return ftrace_set_regex(ops, buf, len, reset, 1);
 }
 EXPORT_SYMBOL_GPL(ftrace_set_filter);
 
@@ -2957,10 +2960,10 @@ EXPORT_SYMBOL_GPL(ftrace_set_filter);
  * is enabled. If @buf is NULL and reset is set, all functions will be enabled
  * for tracing.
  */
-void ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
+int ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
 			int len, int reset)
 {
-	ftrace_set_regex(ops, buf, len, reset, 0);
+	return ftrace_set_regex(ops, buf, len, reset, 0);
 }
 EXPORT_SYMBOL_GPL(ftrace_set_notrace);
 /**
Signature · 1 line
1.7.1
#101 lore raw

[tip:perf/core] ftrace: Change filter/ notrace set functions to return exit code

Commit-ID:  ac483c446b67870444c9eeaf8325d3d2af9b91bc
Gitweb:     http://git.kernel.org/tip/ac483c446b67870444c9eeaf8325d3d2af9b91bc
Author:     Jiri Olsa <[email protected]>
AuthorDate: Mon, 2 Jan 2012 10:04:14 +0100
Committer:  Steven Rostedt <[email protected]>
CommitDate: Fri, 3 Feb 2012 09:48:18 -0500

ftrace: Change filter/notrace set functions to return exit code

Currently the ftrace_set_filter and ftrace_set_notrace functions do not return any return code. So there's no way for ftrace_ops user to tell wether the filter was correctly applied.

The set_ftrace_filter interface returns error in case the filter did not match:

  # echo krava > set_ftrace_filter
  bash: echo: write error: Invalid argument

Changing both ftrace_set_filter and ftrace_set_notrace functions to return zero if the filter was applied correctly or -E* values in case of error.

---
 include/linux/ftrace.h |    4 ++--
 kernel/trace/ftrace.c  |   15 +++++++++------
 2 files changed, 11 insertions(+), 8 deletions(-)
include/linux/ftrace.h +2 -2
index 028e26f..f33fb3b 100644
@@ -178,9 +178,9 @@ struct dyn_ftrace {
 };
 
 int ftrace_force_update(void);
-void ftrace_set_filter(struct ftrace_ops *ops, unsigned char *buf,
+int ftrace_set_filter(struct ftrace_ops *ops, unsigned char *buf,
 		       int len, int reset);
-void ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
+int ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
 			int len, int reset);
 void ftrace_set_global_filter(unsigned char *buf, int len, int reset);
 void ftrace_set_global_notrace(unsigned char *buf, int len, int reset);
kernel/trace/ftrace.c +9 -6
index 683d559..e2e0597 100644
@@ -3146,8 +3146,10 @@ ftrace_set_regex(struct ftrace_ops *ops, unsigned char *buf, int len,
 	mutex_lock(&ftrace_regex_lock);
 	if (reset)
 		ftrace_filter_reset(hash);
-	if (buf)
-		ftrace_match_records(hash, buf, len);
+	if (buf && !ftrace_match_records(hash, buf, len)) {
+		ret = -EINVAL;
+		goto out_regex_unlock;
+	}
 
 	mutex_lock(&ftrace_lock);
 	ret = ftrace_hash_move(ops, enable, orig_hash, hash);
@@ -3157,6 +3159,7 @@ ftrace_set_regex(struct ftrace_ops *ops, unsigned char *buf, int len,
 
 	mutex_unlock(&ftrace_lock);
 
+ out_regex_unlock:
 	mutex_unlock(&ftrace_regex_lock);
 
 	free_ftrace_hash(hash);
@@ -3173,10 +3176,10 @@ ftrace_set_regex(struct ftrace_ops *ops, unsigned char *buf, int len,
  * Filters denote which functions should be enabled when tracing is enabled.
  * If @buf is NULL and reset is set, all functions will be enabled for tracing.
  */
-void ftrace_set_filter(struct ftrace_ops *ops, unsigned char *buf,
+int ftrace_set_filter(struct ftrace_ops *ops, unsigned char *buf,
 		       int len, int reset)
 {
-	ftrace_set_regex(ops, buf, len, reset, 1);
+	return ftrace_set_regex(ops, buf, len, reset, 1);
 }
 EXPORT_SYMBOL_GPL(ftrace_set_filter);
 
@@ -3191,10 +3194,10 @@ EXPORT_SYMBOL_GPL(ftrace_set_filter);
  * is enabled. If @buf is NULL and reset is set, all functions will be enabled
  * for tracing.
  */
-void ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
+int ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
 			int len, int reset)
 {
-	ftrace_set_regex(ops, buf, len, reset, 0);
+	return ftrace_set_regex(ops, buf, len, reset, 0);
 }
 EXPORT_SYMBOL_GPL(ftrace_set_notrace);
 /**
author
#102 lore raw

PATCH v5 2/7 ftrace: Add enable/disable ftrace_ops control interface

In reply to Jiri Olsa

Adding a way to temporarily enable/disable ftrace_ops. The change follows the same way as 'global' ftrace_ops are done.

Introducing 2 global ftrace_ops - control_ops and ftrace_control_list which take over all ftrace_ops registered with FTRACE_OPS_FL_CONTROL flag. In addition new per cpu flag called 'disabled' is also added to ftrace_ops to provide the control information for each cpu.

When ftrace_ops with FTRACE_OPS_FL_CONTROL is registered, it is set as disabled for all cpus.

The ftrace_control_list contains all the registered 'control' ftrace_ops. The control_ops provides function which iterates ftrace_control_list and does the check for 'disabled' flag on current cpu.

Adding 2 inline functions ftrace_function_enable/ftrace_function_disable, which enable/disable the ftrace_ops for given cpu.

---
 include/linux/ftrace.h |   42 +++++++++++++++++
 kernel/trace/ftrace.c  |  119 +++++++++++++++++++++++++++++++++++++++++++++---
 kernel/trace/trace.h   |    2 +
 3 files changed, 156 insertions(+), 7 deletions(-)
include/linux/ftrace.h +42 -0
index 523640f..0d43a2b 100644
@@ -35,12 +35,14 @@ enum {
 	FTRACE_OPS_FL_ENABLED		= 1 << 0,
 	FTRACE_OPS_FL_GLOBAL		= 1 << 1,
 	FTRACE_OPS_FL_DYNAMIC		= 1 << 2,
+	FTRACE_OPS_FL_CONTROL		= 1 << 3,
 };
 
 struct ftrace_ops {
 	ftrace_func_t			func;
 	struct ftrace_ops		*next;
 	unsigned long			flags;
+	void __percpu			*disabled;
 #ifdef CONFIG_DYNAMIC_FTRACE
 	struct ftrace_hash		*notrace_hash;
 	struct ftrace_hash		*filter_hash;
@@ -97,6 +99,46 @@ int register_ftrace_function(struct ftrace_ops *ops);
 int unregister_ftrace_function(struct ftrace_ops *ops);
 void clear_ftrace_function(void);
 
+/**
+ * ftrace_function_enable - enable controlled ftrace_ops on given cpu
+ *
+ * This function enables tracing on given cpu by decreasing
+ * the per cpu control variable.
+ * It must be called with preemption disabled and only on
+ * ftrace_ops registered with FTRACE_OPS_FL_CONTROL.
+ */
+static inline void ftrace_function_enable(struct ftrace_ops *ops, int cpu)
+{
+	atomic_t *disabled;
+
+	if (WARN_ON_ONCE(!(ops->flags & FTRACE_OPS_FL_CONTROL) ||
+			 !preempt_count()))
+		return;
+
+	disabled = per_cpu_ptr(ops->disabled, cpu);
+	atomic_dec(disabled);
+}
+
+/**
+ * ftrace_function_disable - enable controlled ftrace_ops on given cpu
+ *
+ * This function enables tracing on given cpu by decreasing
+ * the per cpu control variable.
+ * It must be called with preemption disabled and only on
+ * ftrace_ops registered with FTRACE_OPS_FL_CONTROL.
+ */
+static inline void ftrace_function_disable(struct ftrace_ops *ops, int cpu)
+{
+	atomic_t *disabled;
+
+	if (WARN_ON_ONCE(!(ops->flags & FTRACE_OPS_FL_CONTROL) ||
+			 !preempt_count()))
+		return;
+
+	disabled = per_cpu_ptr(ops->disabled, cpu);
+	atomic_inc(disabled);
+}
+
 extern void ftrace_stub(unsigned long a0, unsigned long a1);
 
 #else /* !CONFIG_FUNCTION_TRACER */
kernel/trace/ftrace.c +112 -7

174 lines not on this page · raw message

kernel/trace/trace.h +2 -0
index 2c26574..41c54e3 100644
@@ -288,6 +288,8 @@ struct tracer {
 /* for function tracing recursion */
 #define TRACE_INTERNAL_BIT		(1<<11)
 #define TRACE_GLOBAL_BIT		(1<<12)
+#define TRACE_CONTROL_BIT		(1<<13)
+
 /*
  * Abuse of the trace_recursion.
  * As we need a way to maintain state if we are tracing the function
Signature · 1 line
1.7.1
#103 lore raw

Re: [PATCH 2/7] ftrace: Add enable/disable ftrace_ops control interface

Quoted 15 lines · Jiri Olsa

Adding a way to temporarily enable/disable ftrace_ops. The change follows the same way as 'global' ftrace_ops are done.

Introducing 2 global ftrace_ops - control_ops and ftrace_control_list which take over all ftrace_ops registered with FTRACE_OPS_FL_CONTROL flag. In addition new per cpu flag called 'disabled' is also added to ftrace_ops to provide the control information for each cpu.

When ftrace_ops with FTRACE_OPS_FL_CONTROL is registered, it is set as disabled for all cpus.

The ftrace_control_list contains all the registered 'control' ftrace_ops. The control_ops provides function which iterates ftrace_control_list and does the check for 'disabled' flag on current cpu.

Quoted 5 lines

Adding 2 inline functions ftrace_function_enable/ftrace_function_disable, which enable/disable the ftrace_ops for given cpu.

---

So this is used to implement pmu->add() / -> del(). But perf_tp_event_match() already takes care of that by checking PERF_HES_STOPPED.

Now what you are doing here is an interesting optimization as it doesn't even call on ftrace_ops that have called pmu->del().

I'm not against the idea but I want to ensure this is really your purpose and it would be nice to put some words about that in the changelog as well as in PATCH 4/7.

Quoted 10 lines
 include/linux/ftrace.h |   42 +++++++++++++++++
 kernel/trace/ftrace.c  |  119 +++++++++++++++++++++++++++++++++++++++++++++---
 kernel/trace/trace.h   |    2 +
 3 files changed, 156 insertions(+), 7 deletions(-)
include/linux/ftrace.h +0 -0
index 523640f..0d43a2b 100644

include/linux/ftrace.h enum { · line 38

Quoted 4 lines
include/linux/ftrace.h +1 -0
@@ -35,12 +35,14 @@ enum {
 	FTRACE_OPS_FL_ENABLED		= 1 << 0,
 	FTRACE_OPS_FL_GLOBAL		= 1 << 1,
 	FTRACE_OPS_FL_DYNAMIC		= 1 << 2,
+	FTRACE_OPS_FL_CONTROL		= 1 << 3,

Please comment the role of this flag. In fact it would be nice to have a comment that explains all these. GLOBAL and DYNAMIC don't actually give much clues alone.

Thanks.

maintainer
#104 lore raw

Re: [PATCH 2/7] ftrace: Add enable/disable ftrace_ops control interface

Quoted 14 lines · Frederic Weisbecker

include/linux/ftrace.h enum { · line 38

Quoted 10 lines
include/linux/ftrace.h +1 -0
index 523640f..0d43a2b 100644
@@ -35,12 +35,14 @@ enum {
 	FTRACE_OPS_FL_ENABLED		= 1 << 0,
 	FTRACE_OPS_FL_GLOBAL		= 1 << 1,
 	FTRACE_OPS_FL_DYNAMIC		= 1 << 2,
+	FTRACE_OPS_FL_CONTROL		= 1 << 3,

Please comment the role of this flag. In fact it would be nice to have a comment that explains all these. GLOBAL and DYNAMIC don't actually give much clues alone.

Someone else asked about commenting these (was it you?). I probably should, as it is confusing to what they are used for.

I just figured every just figures it out by looking at the code ;-)

-- Steve

#105 lore raw

Re: [PATCH 2/7] ftrace: Add enable/disable ftrace_ops control interface

Quoted 16 lines · Steven Rostedt
Quoted 14 lines · Frederic Weisbecker

include/linux/ftrace.h enum { · line 38

Quoted 10 lines
include/linux/ftrace.h +1 -0
index 523640f..0d43a2b 100644
@@ -35,12 +35,14 @@ enum {
 	FTRACE_OPS_FL_ENABLED		= 1 << 0,
 	FTRACE_OPS_FL_GLOBAL		= 1 << 1,
 	FTRACE_OPS_FL_DYNAMIC		= 1 << 2,
+	FTRACE_OPS_FL_CONTROL		= 1 << 3,

Please comment the role of this flag. In fact it would be nice to have a comment that explains all these. GLOBAL and DYNAMIC don't actually give much clues alone.

Quoted 1 line

Someone else asked about commenting these (was it you?).

Probably, as I asked you the meaning of that GLOBAL on irc :)

Quoted 1 line

I probably should, as it is confusing to what they are used for.

Yeah that would be nice.

Quoted 1 line

I just figured every just figures it out by looking at the code ;-)

That's often what happens when the reader of the code is also the writer :) Also, there is no user that doesn't have FTRACE_OPS_FL_GLOBAL yet, so it's hard to find a counter example.

author
#106 lore raw

Re: [PATCH 2/7] ftrace: Add enable/disable ftrace_ops control interface

In reply to Frederic Weisbecker
Quoted 25 lines · Frederic Weisbecker
Quoted 15 lines · Jiri Olsa

Adding a way to temporarily enable/disable ftrace_ops. The change follows the same way as 'global' ftrace_ops are done.

Introducing 2 global ftrace_ops - control_ops and ftrace_control_list which take over all ftrace_ops registered with FTRACE_OPS_FL_CONTROL flag. In addition new per cpu flag called 'disabled' is also added to ftrace_ops to provide the control information for each cpu.

When ftrace_ops with FTRACE_OPS_FL_CONTROL is registered, it is set as disabled for all cpus.

The ftrace_control_list contains all the registered 'control' ftrace_ops. The control_ops provides function which iterates ftrace_control_list and does the check for 'disabled' flag on current cpu.

Quoted 5 lines

Adding 2 inline functions ftrace_function_enable/ftrace_function_disable, which enable/disable the ftrace_ops for given cpu.

---

So this is used to implement pmu->add() / -> del(). But perf_tp_event_match() already takes care of that by checking PERF_HES_STOPPED.

Quoted 6 lines

Now what you are doing here is an interesting optimization as it doesn't even call on ftrace_ops that have called pmu->del().

I'm not against the idea but I want to ensure this is really your purpose and it would be nice to put some words about that in the changelog as well as in PATCH 4/7.

well, to say the truth I missed the PERF_HES_STOPPED possibility ;) since I was up to disabling ftrace_ops completely when it's not used..

but as you said it's faster, so I'd stay with it..

I'll try to make some comment about that and repost the patch

Quoted 6 lines
Quoted 6 lines
 include/linux/ftrace.h |   42 +++++++++++++++++
 kernel/trace/ftrace.c  |  119 +++++++++++++++++++++++++++++++++++++++++++++---
 kernel/trace/trace.h   |    2 +
 3 files changed, 156 insertions(+), 7 deletions(-)
Quoted 13 lines

include/linux/ftrace.h enum { · line 38

Quoted 9 lines
include/linux/ftrace.h +1 -0
index 523640f..0d43a2b 100644
@@ -35,12 +35,14 @@ enum {
 	FTRACE_OPS_FL_ENABLED		= 1 << 0,
 	FTRACE_OPS_FL_GLOBAL		= 1 << 1,
 	FTRACE_OPS_FL_DYNAMIC		= 1 << 2,
+	FTRACE_OPS_FL_CONTROL		= 1 << 3,

Please comment the role of this flag. In fact it would be nice to have a comment that explains all these. GLOBAL and DYNAMIC don't actually give much clues alone.

I'll comment those as well..

thanks,
jirka

author
#107 lore raw

PATCH v5 3/7 ftrace, perf: Add open/close tracepoint perf registration actions

In reply to Jiri Olsa

Adding TRACE_REG_PERF_OPEN and TRACE_REG_PERF_CLOSE to differentiate register/unregister from open/close actions.

The register/unregister actions are invoked for the first/last tracepoint user when opening/closing the evet.

The open/close actions are invoked for each tracepoint user when opening/closing the event.

---
 include/linux/ftrace_event.h    |    6 +-
 kernel/trace/trace.h            |    5 ++
 kernel/trace/trace_event_perf.c |  116 +++++++++++++++++++++++++--------------
 kernel/trace/trace_events.c     |   10 ++-
 kernel/trace/trace_kprobe.c     |    6 ++-
 kernel/trace/trace_syscalls.c   |   14 +++-
 6 files changed, 106 insertions(+), 51 deletions(-)
include/linux/ftrace_event.h +4 -2
index c3da42d..195e360 100644
@@ -146,6 +146,8 @@ enum trace_reg {
 	TRACE_REG_UNREGISTER,
 	TRACE_REG_PERF_REGISTER,
 	TRACE_REG_PERF_UNREGISTER,
+	TRACE_REG_PERF_OPEN,
+	TRACE_REG_PERF_CLOSE,
 };
 
 struct ftrace_event_call;
@@ -157,7 +159,7 @@ struct ftrace_event_class {
 	void			*perf_probe;
 #endif
 	int			(*reg)(struct ftrace_event_call *event,
-				       enum trace_reg type);
+				       enum trace_reg type, void *data);
 	int			(*define_fields)(struct ftrace_event_call *);
 	struct list_head	*(*get_fields)(struct ftrace_event_call *);
 	struct list_head	fields;
@@ -165,7 +167,7 @@ struct ftrace_event_class {
 };
 
 extern int ftrace_event_reg(struct ftrace_event_call *event,
-			    enum trace_reg type);
+			    enum trace_reg type, void *data);
 
 enum {
 	TRACE_EVENT_FL_ENABLED_BIT,
kernel/trace/trace.h +5 -0
index 41c54e3..85732a8 100644
@@ -828,4 +828,9 @@ extern const char *__stop___trace_bprintk_fmt[];
 	FTRACE_ENTRY(call, struct_name, id, PARAMS(tstruct), PARAMS(print))
 #include "trace_entries.h"
 
+#ifdef CONFIG_PERF_EVENTS
+int perf_ftrace_event_register(struct ftrace_event_call *call,
+			       enum trace_reg type, void *data);
+#endif
+
 #endif /* _LINUX_KERNEL_TRACE_H */
kernel/trace/trace_event_perf.c +75 -41

154 lines not on this page · raw message

kernel/trace/trace_events.c +7 -3
index c212a7f..5138fea 100644
@@ -147,7 +147,8 @@ int trace_event_raw_init(struct ftrace_event_call *call)
 }
 EXPORT_SYMBOL_GPL(trace_event_raw_init);
 
-int ftrace_event_reg(struct ftrace_event_call *call, enum trace_reg type)
+int ftrace_event_reg(struct ftrace_event_call *call,
+		     enum trace_reg type, void *data)
 {
 	switch (type) {
 	case TRACE_REG_REGISTER:
@@ -170,6 +171,9 @@ int ftrace_event_reg(struct ftrace_event_call *call, enum trace_reg type)
 					    call->class->perf_probe,
 					    call);
 		return 0;
+	case TRACE_REG_PERF_OPEN:
+	case TRACE_REG_PERF_CLOSE:
+		return 0;
 #endif
 	}
 	return 0;
@@ -209,7 +213,7 @@ static int ftrace_event_enable_disable(struct ftrace_event_call *call,
 				tracing_stop_cmdline_record();
 				call->flags &= ~TRACE_EVENT_FL_RECORDED_CMD;
 			}
-			call->class->reg(call, TRACE_REG_UNREGISTER);
+			call->class->reg(call, TRACE_REG_UNREGISTER, NULL);
 		}
 		break;
 	case 1:
@@ -218,7 +222,7 @@ static int ftrace_event_enable_disable(struct ftrace_event_call *call,
 				tracing_start_cmdline_record();
 				call->flags |= TRACE_EVENT_FL_RECORDED_CMD;
 			}
-			ret = call->class->reg(call, TRACE_REG_REGISTER);
+			ret = call->class->reg(call, TRACE_REG_REGISTER, NULL);
 			if (ret) {
 				tracing_stop_cmdline_record();
 				pr_info("event trace: Could not enable event "
kernel/trace/trace_kprobe.c +5 -1
index 00d527c..5667f89 100644
@@ -1892,7 +1892,8 @@ static __kprobes void kretprobe_perf_func(struct kretprobe_instance *ri,
 #endif	/* CONFIG_PERF_EVENTS */
 
 static __kprobes
-int kprobe_register(struct ftrace_event_call *event, enum trace_reg type)
+int kprobe_register(struct ftrace_event_call *event,
+		    enum trace_reg type, void *data)
 {
 	struct trace_probe *tp = (struct trace_probe *)event->data;
 
@@ -1909,6 +1910,9 @@ int kprobe_register(struct ftrace_event_call *event, enum trace_reg type)
 	case TRACE_REG_PERF_UNREGISTER:
 		disable_trace_probe(tp, TP_FLAG_PROFILE);
 		return 0;
+	case TRACE_REG_PERF_OPEN:
+	case TRACE_REG_PERF_CLOSE:
+		return 0;
 #endif
 	}
 	return 0;
kernel/trace/trace_syscalls.c +10 -4
index 5f35f6f..8599c1d 100644
@@ -18,9 +18,9 @@ static DECLARE_BITMAP(enabled_enter_syscalls, NR_syscalls);
 static DECLARE_BITMAP(enabled_exit_syscalls, NR_syscalls);
 
 static int syscall_enter_register(struct ftrace_event_call *event,
-				 enum trace_reg type);
+				 enum trace_reg type, void *data);
 static int syscall_exit_register(struct ftrace_event_call *event,
-				 enum trace_reg type);
+				 enum trace_reg type, void *data);
 
 static int syscall_enter_define_fields(struct ftrace_event_call *call);
 static int syscall_exit_define_fields(struct ftrace_event_call *call);
@@ -650,7 +650,7 @@ void perf_sysexit_disable(struct ftrace_event_call *call)
 #endif /* CONFIG_PERF_EVENTS */
 
 static int syscall_enter_register(struct ftrace_event_call *event,
-				 enum trace_reg type)
+				 enum trace_reg type, void *data)
 {
 	switch (type) {
 	case TRACE_REG_REGISTER:
@@ -665,13 +665,16 @@ static int syscall_enter_register(struct ftrace_event_call *event,
 	case TRACE_REG_PERF_UNREGISTER:
 		perf_sysenter_disable(event);
 		return 0;
+	case TRACE_REG_PERF_OPEN:
+	case TRACE_REG_PERF_CLOSE:
+		return 0;
 #endif
 	}
 	return 0;
 }
 
 static int syscall_exit_register(struct ftrace_event_call *event,
-				 enum trace_reg type)
+				 enum trace_reg type, void *data)
 {
 	switch (type) {
 	case TRACE_REG_REGISTER:
@@ -686,6 +689,9 @@ static int syscall_exit_register(struct ftrace_event_call *event,
 	case TRACE_REG_PERF_UNREGISTER:
 		perf_sysexit_disable(event);
 		return 0;
+	case TRACE_REG_PERF_OPEN:
+	case TRACE_REG_PERF_CLOSE:
+		return 0;
 #endif
 	}
 	return 0;
Signature · 1 line
1.7.1
author
#108 lore raw

PATCH v5 4/7 ftrace, perf: Add add/del tracepoint perf registration actions

In reply to Jiri Olsa

Adding TRACE_REG_PERF_ADD and TRACE_REG_PERF_DEL to handle perf event schedule in/out actions.

The add action is invoked for when the perf event is scheduled in, while the del action is invoked when the event is scheduled out.

---
 include/linux/ftrace_event.h    |    2 ++
 kernel/trace/trace_event_perf.c |    4 +++-
 kernel/trace/trace_events.c     |    2 ++
 kernel/trace/trace_kprobe.c     |    2 ++
 kernel/trace/trace_syscalls.c   |    4 ++++
 5 files changed, 13 insertions(+), 1 deletions(-)
include/linux/ftrace_event.h +2 -0
index 195e360..2bf677c 100644
@@ -148,6 +148,8 @@ enum trace_reg {
 	TRACE_REG_PERF_UNREGISTER,
 	TRACE_REG_PERF_OPEN,
 	TRACE_REG_PERF_CLOSE,
+	TRACE_REG_PERF_ADD,
+	TRACE_REG_PERF_DEL,
 };
 
 struct ftrace_event_call;
kernel/trace/trace_event_perf.c +3 -1
index 0cfcc37..d72af0b 100644
@@ -211,12 +211,14 @@ int perf_trace_add(struct perf_event *p_event, int flags)
 	list = this_cpu_ptr(pcpu_list);
 	hlist_add_head_rcu(&p_event->hlist_entry, list);
 
-	return 0;
+	return tp_event->class->reg(tp_event, TRACE_REG_PERF_ADD, p_event);
 }
 
 void perf_trace_del(struct perf_event *p_event, int flags)
 {
+	struct ftrace_event_call *tp_event = p_event->tp_event;
 	hlist_del_rcu(&p_event->hlist_entry);
+	tp_event->class->reg(tp_event, TRACE_REG_PERF_DEL, p_event);
 }
 
 __kprobes void *perf_trace_buf_prepare(int size, unsigned short type,
kernel/trace/trace_events.c +2 -0
index 5138fea..079a93a 100644
@@ -173,6 +173,8 @@ int ftrace_event_reg(struct ftrace_event_call *call,
 		return 0;
 	case TRACE_REG_PERF_OPEN:
 	case TRACE_REG_PERF_CLOSE:
+	case TRACE_REG_PERF_ADD:
+	case TRACE_REG_PERF_DEL:
 		return 0;
 #endif
 	}
kernel/trace/trace_kprobe.c +2 -0
index 5667f89..580a05e 100644
@@ -1912,6 +1912,8 @@ int kprobe_register(struct ftrace_event_call *event,
 		return 0;
 	case TRACE_REG_PERF_OPEN:
 	case TRACE_REG_PERF_CLOSE:
+	case TRACE_REG_PERF_ADD:
+	case TRACE_REG_PERF_DEL:
 		return 0;
 #endif
 	}
kernel/trace/trace_syscalls.c +4 -0
index 8599c1d..5e4f62e 100644
@@ -667,6 +667,8 @@ static int syscall_enter_register(struct ftrace_event_call *event,
 		return 0;
 	case TRACE_REG_PERF_OPEN:
 	case TRACE_REG_PERF_CLOSE:
+	case TRACE_REG_PERF_ADD:
+	case TRACE_REG_PERF_DEL:
 		return 0;
 #endif
 	}
@@ -691,6 +693,8 @@ static int syscall_exit_register(struct ftrace_event_call *event,
 		return 0;
 	case TRACE_REG_PERF_OPEN:
 	case TRACE_REG_PERF_CLOSE:
+	case TRACE_REG_PERF_ADD:
+	case TRACE_REG_PERF_DEL:
 		return 0;
 #endif
 	}
Signature · 1 line
1.7.1
author
#109 lore raw

PATCH v5 5/7 ftrace, perf: Add support to use function tracepoint in perf

In reply to Jiri Olsa

Adding perf registration support for the ftrace function event, so it is now possible to register it via perf interface.

The perf_event struct statically contains ftrace_ops as a handle for function tracer. The function tracer is registered/unregistered in open/close actions, and enabled/disabled in add/del actions.

It is now possible to use function trace within perf commands like:

perf record -e ftrace:function ls
perf stat -e ftrace:function ls

Allowed only for root.

---
 include/linux/perf_event.h      |    3 +
 kernel/trace/trace.h            |    2 +
 kernel/trace/trace_event_perf.c |   92 +++++++++++++++++++++++++++++++++++++++
 kernel/trace/trace_export.c     |   28 ++++++++++++
 4 files changed, 125 insertions(+), 0 deletions(-)
include/linux/perf_event.h +3 -0
index 0b91db2..5003be6 100644
@@ -858,6 +858,9 @@ struct perf_event {
 #ifdef CONFIG_EVENT_TRACING
 	struct ftrace_event_call	*tp_event;
 	struct event_filter		*filter;
+#ifdef CONFIG_FUNCTION_TRACER
+	struct ftrace_ops               ftrace_ops;
+#endif
 #endif
 
 #ifdef CONFIG_CGROUP_PERF
kernel/trace/trace.h +2 -0
index 85732a8..e88e58a 100644
@@ -591,6 +591,8 @@ static inline int ftrace_trace_task(struct task_struct *task)
 static inline int ftrace_is_dead(void) { return 0; }
 #endif
 
+int ftrace_event_is_function(struct ftrace_event_call *call);
+
 /*
  * struct trace_parser - servers for reading the user input separated by spaces
  * @cont: set if the input is not complete - no final space char was found
kernel/trace/trace_event_perf.c +92 -0

101 lines not on this page · raw message

kernel/trace/trace_export.c +28 -0
index bbeec31..867653c 100644
@@ -131,6 +131,28 @@ ftrace_define_fields_##name(struct ftrace_event_call *event_call)	\
 
 #include "trace_entries.h"
 
+static int ftrace_event_class_register(struct ftrace_event_call *call,
+				       enum trace_reg type, void *data)
+{
+	switch (type) {
+	case TRACE_REG_PERF_REGISTER:
+	case TRACE_REG_PERF_UNREGISTER:
+		return 0;
+	case TRACE_REG_PERF_OPEN:
+	case TRACE_REG_PERF_CLOSE:
+	case TRACE_REG_PERF_ADD:
+	case TRACE_REG_PERF_DEL:
+#ifdef CONFIG_PERF_EVENTS
+		return perf_ftrace_event_register(call, type, data);
+#endif
+	case TRACE_REG_REGISTER:
+	case TRACE_REG_UNREGISTER:
+		break;
+	}
+
+	return -EINVAL;
+}
+
 #undef __entry
 #define __entry REC
 
@@ -159,6 +181,7 @@ struct ftrace_event_class event_class_ftrace_##call = {			\
 	.system			= __stringify(TRACE_SYSTEM),		\
 	.define_fields		= ftrace_define_fields_##call,		\
 	.fields			= LIST_HEAD_INIT(event_class_ftrace_##call.fields),\
+	.reg			= ftrace_event_class_register,		\
 };									\
 									\
 struct ftrace_event_call __used event_##call = {			\
@@ -170,4 +193,9 @@ struct ftrace_event_call __used event_##call = {			\
 struct ftrace_event_call __used						\
 __attribute__((section("_ftrace_events"))) *__event_##call = &event_##call;
 
+int ftrace_event_is_function(struct ftrace_event_call *call)
+{
+	return call == &event_function;
+}
+
 #include "trace_entries.h"
Signature · 1 line
1.7.1
author
#110 lore raw

PATCH v5 6/7 ftrace, perf: Distinguish ftrace function event field type

In reply to Jiri Olsa

Adding FILTER_TRACE_FN event field type for function tracepoint event, so it can be properly recognized within filtering code.

Currently all fields of ftrace subsystem events share the common field type FILTER_OTHER. Since the function trace fields need special care within the filtering code we need to recognize it properly, hence adding the FILTER_TRACE_FN event type.

---
 include/linux/ftrace_event.h       |    1 +
 kernel/trace/trace_events_filter.c |    7 ++++++-
 kernel/trace/trace_export.c        |   25 ++++++++++++++++++++-----
 3 files changed, 27 insertions(+), 6 deletions(-)
include/linux/ftrace_event.h +1 -0
index 2bf677c..dd478fc 100644
@@ -245,6 +245,7 @@ enum {
 	FILTER_STATIC_STRING,
 	FILTER_DYN_STRING,
 	FILTER_PTR_STRING,
+	FILTER_TRACE_FN,
 };
 
 #define EVENT_STORAGE_SIZE 128
kernel/trace/trace_events_filter.c +6 -1
index f04cc31..66b74ab 100644
@@ -900,6 +900,11 @@ int filter_assign_type(const char *type)
 	return FILTER_OTHER;
 }
 
+static bool is_function_field(struct ftrace_event_field *field)
+{
+	return field->filter_type == FILTER_TRACE_FN;
+}
+
 static bool is_string_field(struct ftrace_event_field *field)
 {
 	return field->filter_type == FILTER_DYN_STRING ||
@@ -987,7 +992,7 @@ static int init_pred(struct filter_parse_state *ps,
 			fn = filter_pred_strloc;
 		else
 			fn = filter_pred_pchar;
-	} else {
+	} else if (!is_function_field(field)) {
 		if (field->is_signed)
 			ret = strict_strtoll(pred->regex.pattern, 0, &val);
 		else
kernel/trace/trace_export.c +20 -5
index 867653c..46c35e2 100644
@@ -67,7 +67,7 @@ static void __always_unused ____ftrace_check_##name(void)	\
 	ret = trace_define_field(event_call, #type, #item,		\
 				 offsetof(typeof(field), item),		\
 				 sizeof(field.item),			\
-				 is_signed_type(type), FILTER_OTHER);	\
+				 is_signed_type(type), filter_type);	\
 	if (ret)							\
 		return ret;
 
@@ -77,7 +77,7 @@ static void __always_unused ____ftrace_check_##name(void)	\
 				 offsetof(typeof(field),		\
 					  container.item),		\
 				 sizeof(field.container.item),		\
-				 is_signed_type(type), FILTER_OTHER);	\
+				 is_signed_type(type), filter_type);	\
 	if (ret)							\
 		return ret;
 
@@ -91,7 +91,7 @@ static void __always_unused ____ftrace_check_##name(void)	\
 		ret = trace_define_field(event_call, event_storage, #item, \
 				 offsetof(typeof(field), item),		\
 				 sizeof(field.item),			\
-				 is_signed_type(type), FILTER_OTHER);	\
+				 is_signed_type(type), filter_type);	\
 		mutex_unlock(&event_storage_mutex);			\
 		if (ret)						\
 			return ret;					\
@@ -104,7 +104,7 @@ static void __always_unused ____ftrace_check_##name(void)	\
 				 offsetof(typeof(field),		\
 					  container.item),		\
 				 sizeof(field.container.item),		\
-				 is_signed_type(type), FILTER_OTHER);	\
+				 is_signed_type(type), filter_type);	\
 	if (ret)							\
 		return ret;
 
@@ -112,10 +112,24 @@ static void __always_unused ____ftrace_check_##name(void)	\
 #define __dynamic_array(type, item)					\
 	ret = trace_define_field(event_call, #type, #item,		\
 				 offsetof(typeof(field), item),		\
-				 0, is_signed_type(type), FILTER_OTHER);\
+				 0, is_signed_type(type), filter_type);\
 	if (ret)							\
 		return ret;
 
+#define FILTER_TYPE_TRACE_FN           FILTER_TRACE_FN
+#define FILTER_TYPE_TRACE_GRAPH_ENT    FILTER_OTHER
+#define FILTER_TYPE_TRACE_GRAPH_RET    FILTER_OTHER
+#define FILTER_TYPE_TRACE_CTX          FILTER_OTHER
+#define FILTER_TYPE_TRACE_WAKE         FILTER_OTHER
+#define FILTER_TYPE_TRACE_STACK                FILTER_OTHER
+#define FILTER_TYPE_TRACE_USER_STACK   FILTER_OTHER
+#define FILTER_TYPE_TRACE_BPRINT       FILTER_OTHER
+#define FILTER_TYPE_TRACE_PRINT                FILTER_OTHER
+#define FILTER_TYPE_TRACE_MMIO_RW      FILTER_OTHER
+#define FILTER_TYPE_TRACE_MMIO_MAP     FILTER_OTHER
+#define FILTER_TYPE_TRACE_BRANCH       FILTER_OTHER
+#define FILTER_TYPE(arg)               FILTER_TYPE_##arg
+
 #undef FTRACE_ENTRY
 #define FTRACE_ENTRY(name, struct_name, id, tstruct, print)		\
 int									\
@@ -123,6 +137,7 @@ ftrace_define_fields_##name(struct ftrace_event_call *event_call)	\
 {									\
 	struct struct_name field;					\
 	int ret;							\
+	int filter_type = FILTER_TYPE(id);				\
 									\
 	tstruct;							\
 									\
Signature · 1 line
1.7.1
author
#111 lore raw

PATCH v5 7/7 ftrace, perf: Add filter support for function trace event

In reply to Jiri Olsa

Adding support to filter function trace event via perf interface. It is now possible to use filter interface in the perf tool like:

  perf record -e ftrace:function --filter="(ip == mm_*)" ls

The filter syntax is restricted to the the 'ip' field only, and following operators are accepted '==' '!=' '||', ending up with the filter strings like:

ip == f1[, ]f2 ... || ip != f3[, ]f4 ...

with comma ',' or space ' ' as a function separator. If the space ' ' is used as a separator, the right side of the assignment needs to be enclosed in double quotes '"'.

The '==' operator adds trace filter with same effect as would be added via set_ftrace_filter file.

The '!=' operator adds trace filter with same effect as would be added via set_ftrace_notrace file.

The right side of the '!=', '==' operators is list of functions or regexp. to be added to filter separated by space.

The '||' operator is used for connecting multiple filter definitions together. It is possible to have more than one '==' and '!=' operators within one filter string.

---
 include/linux/ftrace.h             |    1 +
 kernel/trace/ftrace.c              |    6 ++
 kernel/trace/trace.h               |    2 -
 kernel/trace/trace_event_perf.c    |    4 +-
 kernel/trace/trace_events_filter.c |  161 ++++++++++++++++++++++++++++++++++--
 5 files changed, 163 insertions(+), 11 deletions(-)
include/linux/ftrace.h +1 -0
index 0d43a2b..40bf05f 100644
@@ -228,6 +228,7 @@ int ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
 			int len, int reset);
 void ftrace_set_global_filter(unsigned char *buf, int len, int reset);
 void ftrace_set_global_notrace(unsigned char *buf, int len, int reset);
+void ftrace_free_filter(struct ftrace_ops *ops);
 
 int register_ftrace_command(struct ftrace_func_command *cmd);
 int unregister_ftrace_command(struct ftrace_func_command *cmd);
kernel/trace/ftrace.c +6 -0
index 7af5fb3..693df34 100644
@@ -1193,6 +1193,12 @@ static void free_ftrace_hash_rcu(struct ftrace_hash *hash)
 	call_rcu_sched(&hash->rcu, __free_ftrace_hash_rcu);
 }
 
+void ftrace_free_filter(struct ftrace_ops *ops)
+{
+	free_ftrace_hash(ops->filter_hash);
+	free_ftrace_hash(ops->notrace_hash);
+}
+
 static struct ftrace_hash *alloc_ftrace_hash(int size_bits)
 {
 	struct ftrace_hash *hash;
kernel/trace/trace.h +0 -2
index e88e58a..4ec6d18 100644
@@ -770,9 +770,7 @@ struct filter_pred {
 	u64 			val;
 	struct regex		regex;
 	unsigned short		*ops;
-#ifdef CONFIG_FTRACE_STARTUP_TEST
 	struct ftrace_event_field *field;
-#endif
 	int 			offset;
 	int 			not;
 	int 			op;
kernel/trace/trace_event_perf.c +3 -1
index 57eb232..220b50a 100644
@@ -298,7 +298,9 @@ static int perf_ftrace_function_register(struct perf_event *event)
 static int perf_ftrace_function_unregister(struct perf_event *event)
 {
 	struct ftrace_ops *ops = &event->ftrace_ops;
-	return unregister_ftrace_function(ops);
+	int ret = unregister_ftrace_function(ops);
+	ftrace_free_filter(ops);
+	return ret;
 }
 
 static void perf_ftrace_function_enable(struct perf_event *event)
kernel/trace/trace_events_filter.c +153 -8

221 lines not on this page · raw message

Signature · 1 line
1.7.1
maintainer
#112 lore raw

Re: [PATCH 7/7] ftrace, perf: Add filter support for function trace event

Quoted 22 lines · Jiri Olsa

Adding support to filter function trace event via perf interface. It is now possible to use filter interface in the perf tool like:

  perf record -e ftrace:function --filter="(ip == mm_*)" ls

The filter syntax is restricted to the the 'ip' field only, and following operators are accepted '==' '!=' '||', ending up with the filter strings like:

ip == f1[, ]f2 ... || ip != f3[, ]f4 ...

with comma ',' or space ' ' as a function separator. If the space ' ' is used as a separator, the right side of the assignment needs to be enclosed in double quotes '"'.

The '==' operator adds trace filter with same effect as would be added via set_ftrace_filter file.

The '!=' operator adds trace filter with same effect as would be added via set_ftrace_notrace file.

Quoted 6 lines

The right side of the '!=', '==' operators is list of functions or regexp. to be added to filter separated by space.

The '||' operator is used for connecting multiple filter definitions together. It is possible to have more than one '==' and '!=' operators within one filter string.

Hate to ask you this, but can you rebase this patch against latest tip/perf/core? Things have changed that cause this patch not to apply.

I'll go ahead and test your other 6 patches.

Thanks!

-- Steve

author
#113 lore raw

Re: [PATCH 7/7] ftrace, perf: Add filter support for function trace event

Quoted 26 lines · Steven Rostedt
Quoted 25 lines · Jiri Olsa

Adding support to filter function trace event via perf interface. It is now possible to use filter interface in the perf tool like:

  perf record -e ftrace:function --filter="(ip == mm_*)" ls

The filter syntax is restricted to the the 'ip' field only, and following operators are accepted '==' '!=' '||', ending up with the filter strings like:

ip == f1[, ]f2 ... || ip != f3[, ]f4 ...

with comma ',' or space ' ' as a function separator. If the space ' ' is used as a separator, the right side of the assignment needs to be enclosed in double quotes '"'.

The '==' operator adds trace filter with same effect as would be added via set_ftrace_filter file.

The '!=' operator adds trace filter with same effect as would be added via set_ftrace_notrace file.

The right side of the '!=', '==' operators is list of functions or regexp. to be added to filter separated by space.

Quoted 6 lines
Quoted 3 lines

The '||' operator is used for connecting multiple filter definitions together. It is possible to have more than one '==' and '!=' operators within one filter string.

Hate to ask you this, but can you rebase this patch against latest tip/perf/core? Things have changed that cause this patch not to apply.

yep, attaching rebased patch ;)

thanks,
jirka

---
Adding support to filter function trace event via perf interface. It is now possible to use filter interface in the perf tool like:

  perf record -e ftrace:function --filter="(ip == mm_*)" ls

The filter syntax is restricted to the the 'ip' field only, and following operators are accepted '==' '!=' '||', ending up with the filter strings like:

ip == f1[, ]f2 ... || ip != f3[, ]f4 ...

with comma ',' or space ' ' as a function separator. If the space ' ' is used as a separator, the right side of the assignment needs to be enclosed in double quotes '"'.

The '==' operator adds trace filter with same effect as would be added via set_ftrace_filter file.

The '!=' operator adds trace filter with same effect as would be added via set_ftrace_notrace file.

The right side of the '!=', '==' operators is list of functions or regexp. to be added to filter separated by space.

The '||' operator is used for connecting multiple filter definitions together. It is possible to have more than one '==' and '!=' operators within one filter string.

---
 include/linux/ftrace.h             |    1 +
 kernel/trace/ftrace.c              |    6 ++
 kernel/trace/trace.h               |    2 -
 kernel/trace/trace_event_perf.c    |    4 +-
 kernel/trace/trace_events_filter.c |  169 +++++++++++++++++++++++++++++++++---
 5 files changed, 168 insertions(+), 14 deletions(-)
include/linux/ftrace.h +1 -0
index 2a6d9af..797a5a5 100644
@@ -226,6 +226,7 @@ int ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
 			int len, int reset);
 void ftrace_set_global_filter(unsigned char *buf, int len, int reset);
 void ftrace_set_global_notrace(unsigned char *buf, int len, int reset);
+void ftrace_free_filter(struct ftrace_ops *ops);
 
 int register_ftrace_command(struct ftrace_func_command *cmd);
 int unregister_ftrace_command(struct ftrace_func_command *cmd);
kernel/trace/ftrace.c +6 -0
index 45c9b0c..9935a2a 100644
@@ -1192,6 +1192,12 @@ static void free_ftrace_hash_rcu(struct ftrace_hash *hash)
 	call_rcu_sched(&hash->rcu, __free_ftrace_hash_rcu);
 }
 
+void ftrace_free_filter(struct ftrace_ops *ops)
+{
+	free_ftrace_hash(ops->filter_hash);
+	free_ftrace_hash(ops->notrace_hash);
+}
+
 static struct ftrace_hash *alloc_ftrace_hash(int size_bits)
 {
 	struct ftrace_hash *hash;
kernel/trace/trace.h +0 -2
index e88e58a..4ec6d18 100644
@@ -770,9 +770,7 @@ struct filter_pred {
 	u64 			val;
 	struct regex		regex;
 	unsigned short		*ops;
-#ifdef CONFIG_FTRACE_STARTUP_TEST
 	struct ftrace_event_field *field;
-#endif
 	int 			offset;
 	int 			not;
 	int 			op;
kernel/trace/trace_event_perf.c +3 -1
index 57eb232..220b50a 100644
@@ -298,7 +298,9 @@ static int perf_ftrace_function_register(struct perf_event *event)
 static int perf_ftrace_function_unregister(struct perf_event *event)
 {
 	struct ftrace_ops *ops = &event->ftrace_ops;
-	return unregister_ftrace_function(ops);
+	int ret = unregister_ftrace_function(ops);
+	ftrace_free_filter(ops);
+	return ret;
 }
 
 static void perf_ftrace_function_enable(struct perf_event *event)
kernel/trace/trace_events_filter.c +158 -11

239 lines not on this page · raw message

Signature · 1 line
1.7.1
author
#114 lore raw
In reply to Jiri Olsa

hi,
any feedback?

thanks,
jirka

Quoted 58 lines · Jiri Olsa

hi,
here's another version of perf support for function trace with filter.

attached patches:
1/7 ftrace: Change filter/notrace set functions to return exit code 2/7 ftrace: Add enable/disable ftrace_ops control interface 3/7 ftrace, perf: Add open/close tracepoint perf registration actions 4/7 ftrace, perf: Add add/del tracepoint perf registration actions 5/7 ftrace, perf: Add support to use function tracepoint in perf 6/7 ftrace, perf: Distinguish ftrace function event field type 7/7 ftrace, perf: Add filter support for function trace event

v5 changes:
7/7 - fixed to properly support ',' in filter expressions

v4 changes:
2/7 - FL_GLOBAL_CONTROL changed to FL_GLOBAL_CONTROL_MASK
- changed WARN_ON_ONCE() to include the !preempt_count()
- changed this_cpu_ptr to per_cpu_ptr

ommited Fix possible NULL dereferencing in __ftrace_hash_rec_update (2/8 in v3)

v3 changes:
3/8 - renamed __add/remove_ftrace_ops
- fixed preemtp_enable/recursion_clear order in ftrace_ops_control_func
- renamed/commented API functions - enable/disable_ftrace_function

ommited graph tracer workarounf patch 10/10

v2 changes:
01/10 - keeping the old fix instead of adding hash_has_contents func
I'll send separating patchset for this
02/10 - using different way to avoid the issue (3/9 in v1) 03/10 - using the way proposed by Steven for controling ftrace_ops

         (4/9 in v1)
 06/10 - added check ensuring the ftrace:function event could be used by
         root only (7/9 in v1)

08/10 - added more description (8/9 in v1)
09/10 - changed '&&' operator to '||' which seems more suitable
in this case (9/9 in v1)

thanks,
jirka

---
 include/linux/ftrace.h             |   47 ++++++++-
 include/linux/ftrace_event.h       |    9 +-
 include/linux/perf_event.h         |    3 +
 kernel/trace/ftrace.c              |  140 +++++++++++++++++++++--
 kernel/trace/trace.h               |   11 ++-
 kernel/trace/trace_event_perf.c    |  214 +++++++++++++++++++++++++++++-------
 kernel/trace/trace_events.c        |   12 ++-
 kernel/trace/trace_events_filter.c |  164 ++++++++++++++++++++++++++-
 kernel/trace/trace_export.c        |   53 ++++++++-
 kernel/trace/trace_kprobe.c        |    8 +-
 kernel/trace/trace_syscalls.c      |   18 +++-
 11 files changed, 598 insertions(+), 81 deletions(-)
maintainer
#115 lore raw
Quoted 3 lines · Jiri Olsa

hi,
any feedback?

Sorry,

Got lost in the after break email. Will look at it this week though.

Thanks,

-- Steve

author
#116 lore raw

v6 cover ftrace, perf: Adding support to use function trace

In reply to Jiri Olsa

hi,
here's another version of perf support for function trace with filter.

attached patches:
1/7 ftrace: Change filter/notrace set functions to return exit code 2/7 ftrace: Add enable/disable ftrace_ops control interface 3/7 ftrace, perf: Add open/close tracepoint perf registration actions 4/7 ftrace, perf: Add add/del tracepoint perf registration actions 5/7 ftrace, perf: Add support to use function tracepoint in perf 6/7 ftrace, perf: Distinguish ftrace function event field type 7/7 ftrace, perf: Add filter support for function trace event

v6 changes:
2/7 - added comments to FTRACE_OPS_FL_* bits enum
5/7 - added more info to the change log regarding ftrace_ops enable/disable 7/7 - rebased to the latest filter changes

v5 changes:
7/7 - fixed to properly support ',' in filter expressions

v4 changes:
2/7 - FL_GLOBAL_CONTROL changed to FL_GLOBAL_CONTROL_MASK
- changed WARN_ON_ONCE() to include the !preempt_count()
- changed this_cpu_ptr to per_cpu_ptr

ommited Fix possible NULL dereferencing in __ftrace_hash_rec_update (2/8 in v3)

v3 changes:
3/8 - renamed __add/remove_ftrace_ops
- fixed preemtp_enable/recursion_clear order in ftrace_ops_control_func
- renamed/commented API functions - enable/disable_ftrace_function

ommited graph tracer workarounf patch 10/10

v2 changes:
01/10 - keeping the old fix instead of adding hash_has_contents func
I'll send separating patchset for this
02/10 - using different way to avoid the issue (3/9 in v1) 03/10 - using the way proposed by Steven for controling ftrace_ops

         (4/9 in v1)
 06/10 - added check ensuring the ftrace:function event could be used by
         root only (7/9 in v1)

08/10 - added more description (8/9 in v1)
09/10 - changed '&&' operator to '||' which seems more suitable
in this case (9/9 in v1)

thanks,
jirka

---
 include/linux/ftrace.h             |   61 ++++++++++-
 include/linux/ftrace_event.h       |    9 +-
 include/linux/perf_event.h         |    3 +
 kernel/trace/ftrace.c              |  140 +++++++++++++++++++++--
 kernel/trace/trace.h               |   11 ++-
 kernel/trace/trace_event_perf.c    |  214 +++++++++++++++++++++++++++++-------
 kernel/trace/trace_events.c        |   12 ++-
 kernel/trace/trace_events_filter.c |  172 +++++++++++++++++++++++++++--
 kernel/trace/trace_export.c        |   53 ++++++++-
 kernel/trace/trace_kprobe.c        |    8 +-
 kernel/trace/trace_syscalls.c      |   18 +++-
 11 files changed, 617 insertions(+), 84 deletions(-)
author
#117 lore raw

PATCH v5 1/7 ftrace: Change filter/notrace set functions to return exit code

Currently the ftrace_set_filter and ftrace_set_notrace functions do not return any return code. So there's no way for ftrace_ops user to tell wether the filter was correctly applied.

The set_ftrace_filter interface returns error in case the filter did not match:

  # echo krava > set_ftrace_filter
  bash: echo: write error: Invalid argument

Changing both ftrace_set_filter and ftrace_set_notrace functions to return zero if the filter was applied correctly or -E* values in case of error.

---
 include/linux/ftrace.h |    4 ++--
 kernel/trace/ftrace.c  |   15 +++++++++------
 2 files changed, 11 insertions(+), 8 deletions(-)
include/linux/ftrace.h +2 -2
index 028e26f..f33fb3b 100644
@@ -178,9 +178,9 @@ struct dyn_ftrace {
 };
 
 int ftrace_force_update(void);
-void ftrace_set_filter(struct ftrace_ops *ops, unsigned char *buf,
+int ftrace_set_filter(struct ftrace_ops *ops, unsigned char *buf,
 		       int len, int reset);
-void ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
+int ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
 			int len, int reset);
 void ftrace_set_global_filter(unsigned char *buf, int len, int reset);
 void ftrace_set_global_notrace(unsigned char *buf, int len, int reset);
kernel/trace/ftrace.c +9 -6
index 683d559..e2e0597 100644
@@ -3146,8 +3146,10 @@ ftrace_set_regex(struct ftrace_ops *ops, unsigned char *buf, int len,
 	mutex_lock(&ftrace_regex_lock);
 	if (reset)
 		ftrace_filter_reset(hash);
-	if (buf)
-		ftrace_match_records(hash, buf, len);
+	if (buf && !ftrace_match_records(hash, buf, len)) {
+		ret = -EINVAL;
+		goto out_regex_unlock;
+	}
 
 	mutex_lock(&ftrace_lock);
 	ret = ftrace_hash_move(ops, enable, orig_hash, hash);
@@ -3157,6 +3159,7 @@ ftrace_set_regex(struct ftrace_ops *ops, unsigned char *buf, int len,
 
 	mutex_unlock(&ftrace_lock);
 
+ out_regex_unlock:
 	mutex_unlock(&ftrace_regex_lock);
 
 	free_ftrace_hash(hash);
@@ -3173,10 +3176,10 @@ ftrace_set_regex(struct ftrace_ops *ops, unsigned char *buf, int len,
  * Filters denote which functions should be enabled when tracing is enabled.
  * If @buf is NULL and reset is set, all functions will be enabled for tracing.
  */
-void ftrace_set_filter(struct ftrace_ops *ops, unsigned char *buf,
+int ftrace_set_filter(struct ftrace_ops *ops, unsigned char *buf,
 		       int len, int reset)
 {
-	ftrace_set_regex(ops, buf, len, reset, 1);
+	return ftrace_set_regex(ops, buf, len, reset, 1);
 }
 EXPORT_SYMBOL_GPL(ftrace_set_filter);
 
@@ -3191,10 +3194,10 @@ EXPORT_SYMBOL_GPL(ftrace_set_filter);
  * is enabled. If @buf is NULL and reset is set, all functions will be enabled
  * for tracing.
  */
-void ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
+int ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
 			int len, int reset)
 {
-	ftrace_set_regex(ops, buf, len, reset, 0);
+	return ftrace_set_regex(ops, buf, len, reset, 0);
 }
 EXPORT_SYMBOL_GPL(ftrace_set_notrace);
 /**
Signature · 1 line
1.7.1
Acked-by on v5 1/7
#118 lore raw

Re: [PATCH 1/7] ftrace: Change filter/notrace set functions to return exit code

Quoted 7 lines · Jiri Olsa

Currently the ftrace_set_filter and ftrace_set_notrace functions do not return any return code. So there's no way for ftrace_ops user to tell wether the filter was correctly applied.

The set_ftrace_filter interface returns error in case the filter did not match:

Quoted 8 lines
  # echo krava > set_ftrace_filter
  bash: echo: write error: Invalid argument

Changing both ftrace_set_filter and ftrace_set_notrace functions to return zero if the filter was applied correctly or -E* values in case of error.

author
#119 lore raw

PATCH v5 2/7 ftrace: Add enable/disable ftrace_ops control interface

In reply to Jiri Olsa

Adding a way to temporarily enable/disable ftrace_ops. The change follows the same way as 'global' ftrace_ops are done.

Introducing 2 global ftrace_ops - control_ops and ftrace_control_list which take over all ftrace_ops registered with FTRACE_OPS_FL_CONTROL flag. In addition new per cpu flag called 'disabled' is also added to ftrace_ops to provide the control information for each cpu.

When ftrace_ops with FTRACE_OPS_FL_CONTROL is registered, it is set as disabled for all cpus.

The ftrace_control_list contains all the registered 'control' ftrace_ops. The control_ops provides function which iterates ftrace_control_list and does the check for 'disabled' flag on current cpu.

Adding 2 inline functions ftrace_function_enable/ftrace_function_disable, which enable/disable the ftrace_ops for given cpu.

---
 include/linux/ftrace.h |   56 ++++++++++++++++++++++
 kernel/trace/ftrace.c  |  119 +++++++++++++++++++++++++++++++++++++++++++++---
 kernel/trace/trace.h   |    2 +
 3 files changed, 170 insertions(+), 7 deletions(-)
include/linux/ftrace.h +56 -0
index f33fb3b..d3f529c 100644
@@ -31,16 +31,32 @@ ftrace_enable_sysctl(struct ctl_table *table, int write,
 
 typedef void (*ftrace_func_t)(unsigned long ip, unsigned long parent_ip);
 
+/*
+ * FTRACE_OPS_FL_* bits denote the state of ftrace_ops struct and are
+ * set in the flags member.
+ *
+ * ENABLED - set/unset when ftrace_ops is registered/unregistered
+ * GLOBAL  - set manualy by ftrace_ops user to denote the ftrace_ops
+ *           is part of the global tracers sharing the same filter
+ *           via set_ftrace_* debugfs files.
+ * DYNAMIC - set when ftrace_ops is registered to denote dynamically
+ *           allocated ftrace_ops which need special care
+ * CONTROL - set manualy by ftrace_ops user to denote the ftrace_ops
+ *           could be controled by following calls:
+ *           ftrace_function_enable, ftrace_function_disable
+ */
 enum {
 	FTRACE_OPS_FL_ENABLED		= 1 << 0,
 	FTRACE_OPS_FL_GLOBAL		= 1 << 1,
 	FTRACE_OPS_FL_DYNAMIC		= 1 << 2,
+	FTRACE_OPS_FL_CONTROL		= 1 << 3,
 };
 
 struct ftrace_ops {
 	ftrace_func_t			func;
 	struct ftrace_ops		*next;
 	unsigned long			flags;
+	void __percpu			*disabled;
 #ifdef CONFIG_DYNAMIC_FTRACE
 	struct ftrace_hash		*notrace_hash;
 	struct ftrace_hash		*filter_hash;
@@ -97,6 +113,46 @@ int register_ftrace_function(struct ftrace_ops *ops);
 int unregister_ftrace_function(struct ftrace_ops *ops);
 void clear_ftrace_function(void);
 
+/**
+ * ftrace_function_enable - enable controlled ftrace_ops on given cpu
+ *
+ * This function enables tracing on given cpu by decreasing
+ * the per cpu control variable.
+ * It must be called with preemption disabled and only on
+ * ftrace_ops registered with FTRACE_OPS_FL_CONTROL.
+ */
+static inline void ftrace_function_enable(struct ftrace_ops *ops, int cpu)
+{
+	atomic_t *disabled;
+
+	if (WARN_ON_ONCE(!(ops->flags & FTRACE_OPS_FL_CONTROL) ||
+			 !preempt_count()))
+		return;
+
+	disabled = per_cpu_ptr(ops->disabled, cpu);
+	atomic_dec(disabled);
+}
+
+/**
+ * ftrace_function_disable - enable controlled ftrace_ops on given cpu
+ *
+ * This function enables tracing on given cpu by decreasing
+ * the per cpu control variable.
+ * It must be called with preemption disabled and only on
+ * ftrace_ops registered with FTRACE_OPS_FL_CONTROL.
+ */
+static inline void ftrace_function_disable(struct ftrace_ops *ops, int cpu)
+{
+	atomic_t *disabled;
+
+	if (WARN_ON_ONCE(!(ops->flags & FTRACE_OPS_FL_CONTROL) ||
+			 !preempt_count()))
+		return;
+
+	disabled = per_cpu_ptr(ops->disabled, cpu);
+	atomic_inc(disabled);
+}
+
 extern void ftrace_stub(unsigned long a0, unsigned long a1);
 
 #else /* !CONFIG_FUNCTION_TRACER */
kernel/trace/ftrace.c +112 -7

174 lines not on this page · raw message

kernel/trace/trace.h +2 -0
index 2c26574..41c54e3 100644
@@ -288,6 +288,8 @@ struct tracer {
 /* for function tracing recursion */
 #define TRACE_INTERNAL_BIT		(1<<11)
 #define TRACE_GLOBAL_BIT		(1<<12)
+#define TRACE_CONTROL_BIT		(1<<13)
+
 /*
  * Abuse of the trace_recursion.
  * As we need a way to maintain state if we are tracing the function
Signature · 1 line
1.7.1
#120 lore raw

Re: [PATCH 2/7] ftrace: Add enable/disable ftrace_ops control interface

Quoted 37 lines · Jiri Olsa

Adding a way to temporarily enable/disable ftrace_ops. The change follows the same way as 'global' ftrace_ops are done.

Introducing 2 global ftrace_ops - control_ops and ftrace_control_list which take over all ftrace_ops registered with FTRACE_OPS_FL_CONTROL flag. In addition new per cpu flag called 'disabled' is also added to ftrace_ops to provide the control information for each cpu.

When ftrace_ops with FTRACE_OPS_FL_CONTROL is registered, it is set as disabled for all cpus.

The ftrace_control_list contains all the registered 'control' ftrace_ops. The control_ops provides function which iterates ftrace_control_list and does the check for 'disabled' flag on current cpu.

Adding 2 inline functions ftrace_function_enable/ftrace_function_disable, which enable/disable the ftrace_ops for given cpu.

---
 include/linux/ftrace.h |   56 ++++++++++++++++++++++
 kernel/trace/ftrace.c  |  119 +++++++++++++++++++++++++++++++++++++++++++++---
 kernel/trace/trace.h   |    2 +
 3 files changed, 170 insertions(+), 7 deletions(-)
include/linux/ftrace.h +4 -0
index f33fb3b..d3f529c 100644
@@ -31,16 +31,32 @@ ftrace_enable_sysctl(struct ctl_table *table, int write,
 
 typedef void (*ftrace_func_t)(unsigned long ip, unsigned long parent_ip);
 
+/*
+ * FTRACE_OPS_FL_* bits denote the state of ftrace_ops struct and are
+ * set in the flags member.
+ *

include/linux/ftrace.h ftrace_enable_sysctl(struct ctl_table *table, int write, · line 47

Quoted 10 lines
include/linux/ftrace.h +10 -0
+ * ENABLED - set/unset when ftrace_ops is registered/unregistered
+ * GLOBAL  - set manualy by ftrace_ops user to denote the ftrace_ops
+ *           is part of the global tracers sharing the same filter
+ *           via set_ftrace_* debugfs files.
+ * DYNAMIC - set when ftrace_ops is registered to denote dynamically
+ *           allocated ftrace_ops which need special care
+ * CONTROL - set manualy by ftrace_ops user to denote the ftrace_ops
+ *           could be controled by following calls:
+ *           ftrace_function_enable, ftrace_function_disable
+ */

Nice!

But I have some more comments:

Quoted 28 lines
include/linux/ftrace.h +11 -0
 enum {
 	FTRACE_OPS_FL_ENABLED		= 1 << 0,
 	FTRACE_OPS_FL_GLOBAL		= 1 << 1,
 	FTRACE_OPS_FL_DYNAMIC		= 1 << 2,
+	FTRACE_OPS_FL_CONTROL		= 1 << 3,
 };
 
 struct ftrace_ops {
 	ftrace_func_t			func;
 	struct ftrace_ops		*next;
 	unsigned long			flags;
+	void __percpu			*disabled;
 #ifdef CONFIG_DYNAMIC_FTRACE
 	struct ftrace_hash		*notrace_hash;
 	struct ftrace_hash		*filter_hash;
@@ -97,6 +113,46 @@ int register_ftrace_function(struct ftrace_ops *ops);
 int unregister_ftrace_function(struct ftrace_ops *ops);
 void clear_ftrace_function(void);
 
+/**
+ * ftrace_function_enable - enable controlled ftrace_ops on given cpu
+ *
+ * This function enables tracing on given cpu by decreasing
+ * the per cpu control variable.
+ * It must be called with preemption disabled and only on
+ * ftrace_ops registered with FTRACE_OPS_FL_CONTROL.
+ */
+static inline void ftrace_function_enable(struct ftrace_ops *ops, int cpu)

include/linux/ftrace.h int register_ftrace_function(struct ftrace_ops *ops); · line 134

Quoted 10 lines
include/linux/ftrace.h +10 -0
+{
+	atomic_t *disabled;
+
+	if (WARN_ON_ONCE(!(ops->flags & FTRACE_OPS_FL_CONTROL) ||
+			 !preempt_count()))
+		return;
+
+	disabled = per_cpu_ptr(ops->disabled, cpu);
+	atomic_dec(disabled);
+}

As you're using this for the local CPU exclusively, I suggest you rather rename it to "ftrace_function_{dis,en}able_cpu(struct ftrace_ops *ops)" and use __get_cpu_var() that does the preempt check for you.

[...]

Quoted 20 lines
include/linux/ftrace.h +20 -0
+static void control_ops_disable_all(struct ftrace_ops *ops)
+{
+	int cpu;
+
+	for_each_possible_cpu(cpu)
+		atomic_set(per_cpu_ptr(ops->disabled, cpu), 1);
+}
+
+static int control_ops_alloc(struct ftrace_ops *ops)
+{
+	atomic_t *disabled;
+
+	disabled = alloc_percpu(atomic_t);
+	if (!disabled)
+		return -ENOMEM;
+
+	ops->disabled = disabled;
+	control_ops_disable_all(ops);
+	return 0;
+}

include/linux/ftrace.h int register_ftrace_function(struct ftrace_ops *ops);

Quoted 10 lines
include/linux/ftrace.h +10 -0
+
+static void control_ops_free(struct ftrace_ops *ops)
+{
+	free_percpu(ops->disabled);
+}
+
+static int control_ops_is_disabled(struct ftrace_ops *ops, int cpu)
+{
+	atomic_t *disabled = per_cpu_ptr(ops->disabled, cpu);
+	return atomic_read(disabled);

I think this is checked only locally. Better use __get_cpu_var(). Also note atomic_read() doesn't involve an smp barrier.

atomic_inc/dec are smp safe wrt. ordering. But atomic_set() and atomic_read() are not. I believe this is safe because we still have PERF_HES_STOPPED check.

And also it seems we read the value from the same CPU we have set it. So we actually don't need SMP ordering. But then this raise the question of the relevance of using atomic ops. Normal values would do the trick.

[...]

Quoted 13 lines

static void

include/linux/ftrace.h +12 -0
+ftrace_ops_control_func(unsigned long ip, unsigned long parent_ip)
+{
+	struct ftrace_ops *op;
+	int cpu;
+
+	if (unlikely(trace_recursion_test(TRACE_CONTROL_BIT)))
+		return;
+
+	/*
+	 * Some of the ops may be dynamically allocated,
+	 * they must be freed after a synchronize_sched().
+	 */

include/linux/ftrace.h

Quoted 10 lines
include/linux/ftrace.h +10 -0
+	preempt_disable_notrace();
+	trace_recursion_set(TRACE_CONTROL_BIT);
+	cpu = smp_processor_id();
+	op = rcu_dereference_raw(ftrace_control_list);
+	while (op != &ftrace_list_end) {
+		if (!control_ops_is_disabled(op, cpu) &&
+		    ftrace_ops_test(op, ip))
+			op->func(ip, parent_ip);
+
+		op = rcu_dereference_raw(op->next);

Should it be rcu_dereference_sched() ?

include/linux/ftrace.h

Quoted 8 lines
include/linux/ftrace.h +8 -0
+	};
+	trace_recursion_clear(TRACE_CONTROL_BIT);
+	preempt_enable_notrace();
+}
+
+static struct ftrace_ops control_ops = {
+	.func = ftrace_ops_control_func,
+};

So note this patch is optimizing for the off case (when we have called pmu->del()), but at the cost of having an impact in the on case with having at least one level of multiplexing (and two on the worst case if we have ftrace running in parallel but this is enough a corner case that we don't care).

But this is perhaps still a win.

Quoted 21 lines
include/linux/ftrace.h +2 -0
+
+static void
 ftrace_ops_list_func(unsigned long ip, unsigned long parent_ip)
 {
 	struct ftrace_ops *op;
kernel/trace/trace.h +2 -0
index 2c26574..41c54e3 100644
@@ -288,6 +288,8 @@ struct tracer {
 /* for function tracing recursion */
 #define TRACE_INTERNAL_BIT		(1<<11)
 #define TRACE_GLOBAL_BIT		(1<<12)
+#define TRACE_CONTROL_BIT		(1<<13)
+
 /*
  * Abuse of the trace_recursion.
  * As we need a way to maintain state if we are tracing the function
Signature · 1 line
1.7.1
maintainer
#121 lore raw

Re: [PATCH 2/7] ftrace: Add enable/disable ftrace_ops control interface

Quoted 15 lines · Frederic Weisbecker
Quoted 15 lines
include/linux/ftrace.h +14 -0
+/**
+ * ftrace_function_enable - enable controlled ftrace_ops on given cpu
+ *
+ * This function enables tracing on given cpu by decreasing
+ * the per cpu control variable.
+ * It must be called with preemption disabled and only on
+ * ftrace_ops registered with FTRACE_OPS_FL_CONTROL.
+ */
+static inline void ftrace_function_enable(struct ftrace_ops *ops, int cpu)
+{
+	atomic_t *disabled;
+
+	if (WARN_ON_ONCE(!(ops->flags & FTRACE_OPS_FL_CONTROL) ||
+			 !preempt_count()))
Quoted 8 lines

include/linux/ftrace.h int register_ftrace_function(struct ftrace_ops *ops); · line 134

Quoted 5 lines
include/linux/ftrace.h +5 -0
+		return;
+
+	disabled = per_cpu_ptr(ops->disabled, cpu);
+	atomic_dec(disabled);
+}

As you're using this for the local CPU exclusively, I suggest you rather rename it to "ftrace_function_{dis,en}able_cpu(struct ftrace_ops *ops)"

I wonder if "ftrace_function_local_{dis,en}able(ops)" would be better? That would match something like local_irq_disable/enable.

Quoted 1 line

and use __get_cpu_var() that does the preempt check for you.

Hmm, I haven't tried that with allocated per_cpu pointers before. If it works, sure.

Quoted 26 lines

[...]

Quoted 24 lines
kernel/trace/ftrace.c +24 -0
+static void control_ops_disable_all(struct ftrace_ops *ops)
+{
+	int cpu;
+
+	for_each_possible_cpu(cpu)
+		atomic_set(per_cpu_ptr(ops->disabled, cpu), 1);
+}
+
+static int control_ops_alloc(struct ftrace_ops *ops)
+{
+	atomic_t *disabled;
+
+	disabled = alloc_percpu(atomic_t);
+	if (!disabled)
+		return -ENOMEM;
+
+	ops->disabled = disabled;
+	control_ops_disable_all(ops);
+	return 0;
+}
+
+static void control_ops_free(struct ftrace_ops *ops)
+{
+	free_percpu(ops->disabled);
Quoted 8 lines

kernel/trace/ftrace.c static void ftrace_test_stop_func(unsigned long ip, unsigned long parent_ip) · line 204

Quoted 6 lines
kernel/trace/ftrace.c +6 -0
+}
+
+static int control_ops_is_disabled(struct ftrace_ops *ops, int cpu)
+{
+	atomic_t *disabled = per_cpu_ptr(ops->disabled, cpu);
+	return atomic_read(disabled);

I think this is checked only locally. Better use __get_cpu_var().

If it works, sure.

Quoted 1 line

Also note atomic_read() doesn't involve an smp barrier.

None needed, as this should all be done for the same CPU, and preemption disabled.

Quoted 3 lines

atomic_inc/dec are smp safe wrt. ordering. But atomic_set() and atomic_read() are not. I believe this is safe because we still have PERF_HES_STOPPED check.

It should be safe because smp is not involved. We disable/enable function tracing per cpu, and then check per cpu if it is running. The same task will disable or enable it (I believe in the scheduler).

Quoted 3 lines

And also it seems we read the value from the same CPU we have set it. So we actually don't need SMP ordering. But then this raise the question of

Right.

Quoted 1 line

the relevance of using atomic ops. Normal values would do the trick.

Good point. The atomic here isn't needed.

Quoted 28 lines

[...]

Quoted 13 lines

static void

kernel/trace/ftrace.c +12 -0
+ftrace_ops_control_func(unsigned long ip, unsigned long parent_ip)
+{
+	struct ftrace_ops *op;
+	int cpu;
+
+	if (unlikely(trace_recursion_test(TRACE_CONTROL_BIT)))
+		return;
+
+	/*
+	 * Some of the ops may be dynamically allocated,
+	 * they must be freed after a synchronize_sched().
+	 */

kernel/trace/ftrace.c ftrace_ops_test(struct ftrace_ops *ops, unsigned long ip) · line 3971

Quoted 10 lines
kernel/trace/ftrace.c +10 -0
+	preempt_disable_notrace();
+	trace_recursion_set(TRACE_CONTROL_BIT);
+	cpu = smp_processor_id();
+	op = rcu_dereference_raw(ftrace_control_list);
+	while (op != &ftrace_list_end) {
+		if (!control_ops_is_disabled(op, cpu) &&
+		    ftrace_ops_test(op, ip))
+			op->func(ip, parent_ip);
+
+		op = rcu_dereference_raw(op->next);

Should it be rcu_dereference_sched() ?

Quoted 1 line

From the comment posted by Paul McKenney who converted the global_list

ops (that does somewhat the same thing as the control ops here):

/*
* Traverse the ftrace_global_list, invoking all entries. The reason that we
* can use rcu_dereference_raw() is that elements removed from this list
* are simply leaked, so there is no need to interact with a grace-period
* mechanism. The rcu_dereference_raw() calls are needed to handle
* concurrent insertions into the ftrace_global_list.
*
* Silly Alpha and silly pointer-speculation compiler optimizations! */

But then reading the commit he has:

Replace the calls to read_barrier_depends() in ftrace_list_func() with rcu_dereference_raw() to improve readability. The reason that we use rcu_dereference_raw() here is that removed entries are never freed, instead they are simply leaked. This is one of a very few cases where use of rcu_dereference_raw() is the long-term right answer. And I don't yet know of any others. ;-)

Hmm, and I use the rcu_derefrence_raw() in other places in this file, but those places now get freed. Although, I'm a bit nervous in changing these to rcu_dereference_sched, because if CONFIG_DEBUG_LOCK_ALLOC is enabled, then the checks will be done for *every function* called.

We obviously have preemption disabled, or other bad things may happen. Wonder if we really need this? Ftrace itself is a internal checker and not truly a kernel component. It may be "exempt" from theses checks ;-)

I could make the switch and see what overhead this causes. It may live lock the system. It wouldn't be the first time lockdep & ftrace live locked the system. Or made it so unbearably slow. Lockdep and ftrace do not play well together. They both are very intrusive. The two remind me of the United States congress. Where there is two parties trying to take control of everything, but nothing ever gets done. We end up with a grid/live lock in the country/computer.

Quoted 17 lines

kernel/trace/ftrace.c ftrace_ops_test(struct ftrace_ops *ops, unsigned long ip) · line 3979

Quoted 9 lines
kernel/trace/ftrace.c +8 -0
+	};
+	trace_recursion_clear(TRACE_CONTROL_BIT);
+	preempt_enable_notrace();
+}
+
+static struct ftrace_ops control_ops = {
+	.func = ftrace_ops_control_func,
+};

So note this patch is optimizing for the off case (when we have called pmu->del()), but at the cost of having an impact in the on case with having at least one level of multiplexing (and two on the worst case if we have ftrace running in parallel but this is enough a corner case that we don't care).

Quoted 1 line

But this is perhaps still a win.

There's a lot more overhead elsewhere that this shouldn't be an issue.

-- Steve

#122 lore raw

Re: [PATCH 2/7] ftrace: Add enable/disable ftrace_ops control interface

Quoted 25 lines · Steven Rostedt
Quoted 15 lines · Frederic Weisbecker
Quoted 15 lines
include/linux/ftrace.h +14 -0
+/**
+ * ftrace_function_enable - enable controlled ftrace_ops on given cpu
+ *
+ * This function enables tracing on given cpu by decreasing
+ * the per cpu control variable.
+ * It must be called with preemption disabled and only on
+ * ftrace_ops registered with FTRACE_OPS_FL_CONTROL.
+ */
+static inline void ftrace_function_enable(struct ftrace_ops *ops, int cpu)
+{
+	atomic_t *disabled;
+
+	if (WARN_ON_ONCE(!(ops->flags & FTRACE_OPS_FL_CONTROL) ||
+			 !preempt_count()))
Quoted 8 lines

include/linux/ftrace.h int register_ftrace_function(struct ftrace_ops *ops); · line 134

Quoted 5 lines
include/linux/ftrace.h +5 -0
+		return;
+
+	disabled = per_cpu_ptr(ops->disabled, cpu);
+	atomic_dec(disabled);
+}

As you're using this for the local CPU exclusively, I suggest you rather rename it to "ftrace_function_{dis,en}able_cpu(struct ftrace_ops *ops)"

Quoted 2 lines

I wonder if "ftrace_function_local_{dis,en}able(ops)" would be better? That would match something like local_irq_disable/enable.

Good idea.

Quoted 5 lines
Quoted 2 lines

and use __get_cpu_var() that does the preempt check for you.

Hmm, I haven't tried that with allocated per_cpu pointers before. If it works, sure.

Yeah that works. In fact this_cpu_ptr() would do the best. Both this_cpu_ptr() and __get_cpu_var() map to the same internal ops with preemption checks.

Quoted 29 lines
Quoted 17 lines
Quoted 17 lines

static void

kernel/trace/ftrace.c +16 -0
+ftrace_ops_control_func(unsigned long ip, unsigned long parent_ip)
+{
+	struct ftrace_ops *op;
+	int cpu;
+
+	if (unlikely(trace_recursion_test(TRACE_CONTROL_BIT)))
+		return;
+
+	/*
+	 * Some of the ops may be dynamically allocated,
+	 * they must be freed after a synchronize_sched().
+	 */
+	preempt_disable_notrace();
+	trace_recursion_set(TRACE_CONTROL_BIT);
+	cpu = smp_processor_id();
+	op = rcu_dereference_raw(ftrace_control_list);
Quoted 8 lines

kernel/trace/ftrace.c ftrace_ops_test(struct ftrace_ops *ops, unsigned long ip) · line 3971

Quoted 6 lines
kernel/trace/ftrace.c +6 -0
+	while (op != &ftrace_list_end) {
+		if (!control_ops_is_disabled(op, cpu) &&
+		    ftrace_ops_test(op, ip))
+			op->func(ip, parent_ip);
+
+		op = rcu_dereference_raw(op->next);

Should it be rcu_dereference_sched() ?

From the comment posted by Paul McKenney who converted the global_list ops (that does somewhat the same thing as the control ops here):

Quoted 9 lines

/*
* Traverse the ftrace_global_list, invoking all entries. The reason that we
* can use rcu_dereference_raw() is that elements removed from this list
* are simply leaked, so there is no need to interact with a grace-period
* mechanism. The rcu_dereference_raw() calls are needed to handle
* concurrent insertions into the ftrace_global_list.
*
* Silly Alpha and silly pointer-speculation compiler optimizations! */

Ok.

Quoted 17 lines

But then reading the commit he has:

Replace the calls to read_barrier_depends() in ftrace_list_func() with rcu_dereference_raw() to improve readability. The reason that we use rcu_dereference_raw() here is that removed entries are never freed, instead they are simply leaked. This is one of a very few cases where use of rcu_dereference_raw() is the long-term right answer. And I don't yet know of any others. ;-)

Hmm, and I use the rcu_derefrence_raw() in other places in this file, but those places now get freed. Although, I'm a bit nervous in changing these to rcu_dereference_sched, because if CONFIG_DEBUG_LOCK_ALLOC is enabled, then the checks will be done for *every function* called.

Quoted 7 lines

We obviously have preemption disabled, or other bad things may happen. Wonder if we really need this? Ftrace itself is a internal checker and not truly a kernel component. It may be "exempt" from theses checks ;-)

I could make the switch and see what overhead this causes. It may live lock the system. It wouldn't be the first time lockdep & ftrace live locked the system. Or made it so unbearably slow.

Yeah right. We can probably make this exception for function tracing by keeping rcu_dereference_raw(). But now that it involves a grace period, the comment probably should be updated :)

Quoted 5 lines

Lockdep and ftrace do
not play well together. They both are very intrusive. The two remind me of the United States congress. Where there is two parties trying to take control of everything, but nothing ever gets done. We end up with a grid/live lock in the country/computer.

:o)

Quoted 17 lines
Quoted 17 lines

kernel/trace/ftrace.c ftrace_ops_test(struct ftrace_ops *ops, unsigned long ip) · line 3979

Quoted 9 lines
kernel/trace/ftrace.c +8 -0
+	};
+	trace_recursion_clear(TRACE_CONTROL_BIT);
+	preempt_enable_notrace();
+}
+
+static struct ftrace_ops control_ops = {
+	.func = ftrace_ops_control_func,
+};

So note this patch is optimizing for the off case (when we have called pmu->del()), but at the cost of having an impact in the on case with having at least one level of multiplexing (and two on the worst case if we have ftrace running in parallel but this is enough a corner case that we don't care).

Quoted 3 lines
Quoted 1 line

But this is perhaps still a win.

There's a lot more overhead elsewhere that this shouldn't be an issue.

Yeah that looks fine.

author
#123 lore raw

Re: [PATCH 2/7] ftrace: Add enable/disable ftrace_ops control interface

Quoted 26 lines · Frederic Weisbecker
Quoted 25 lines · Steven Rostedt
Quoted 23 lines · Frederic Weisbecker
Quoted 10 lines
(quoted diff) +9 -0
+/**
+ * ftrace_function_enable - enable controlled ftrace_ops on given cpu
+ *
+ * This function enables tracing on given cpu by decreasing
+ * the per cpu control variable.
+ * It must be called with preemption disabled and only on
+ * ftrace_ops registered with FTRACE_OPS_FL_CONTROL.
+ */
+static inline void ftrace_function_enable(struct ftrace_ops *ops, int cpu)

Review

Quoted 10 lines
(quoted diff) +10 -0
+{
+	atomic_t *disabled;
+
+	if (WARN_ON_ONCE(!(ops->flags & FTRACE_OPS_FL_CONTROL) ||
+			 !preempt_count()))
+		return;
+
+	disabled = per_cpu_ptr(ops->disabled, cpu);
+	atomic_dec(disabled);
+}

As you're using this for the local CPU exclusively, I suggest you rather rename it to "ftrace_function_{dis,en}able_cpu(struct ftrace_ops *ops)"

Quoted 7 lines
Quoted 2 lines

I wonder if "ftrace_function_local_{dis,en}able(ops)" would be better? That would match something like local_irq_disable/enable.

Good idea.

Quoted 2 lines
Quoted 2 lines

and use __get_cpu_var() that does the preempt check for you.

I haven't found preempt check this_cpu_ptr path.. not sure I missed it.. so I'm keeping the implicit preemt check.
Attaching new version of the patch (I'll need to resend whole serie once this one is agreed).

changes:
- using this_cpu_ptr to touch ftrace_ops::disable
- renamed ftrace_ops:disable API:
void ftrace_function_local_enable(struct ftrace_ops *ops) void ftrace_function_local_disable(struct ftrace_ops *ops) int ftrace_function_local_disabled(struct ftrace_ops *ops)

thanks,
jirka

---

include/linux/ftrace.h +66 -0
index f33fb3b..63447bf 100644
@@ -31,16 +31,32 @@ ftrace_enable_sysctl(struct ctl_table *table, int write,
 
 typedef void (*ftrace_func_t)(unsigned long ip, unsigned long parent_ip);
 
+/*
+ * FTRACE_OPS_FL_* bits denote the state of ftrace_ops struct and are
+ * set in the flags member.
+ *
+ * ENABLED - set/unset when ftrace_ops is registered/unregistered
+ * GLOBAL  - set manualy by ftrace_ops user to denote the ftrace_ops
+ *           is part of the global tracers sharing the same filter
+ *           via set_ftrace_* debugfs files.
+ * DYNAMIC - set when ftrace_ops is registered to denote dynamically
+ *           allocated ftrace_ops which need special care
+ * CONTROL - set manualy by ftrace_ops user to denote the ftrace_ops
+ *           could be controled by following calls:
+ *           ftrace_function_enable, ftrace_function_disable
+ */
 enum {
 	FTRACE_OPS_FL_ENABLED		= 1 << 0,
 	FTRACE_OPS_FL_GLOBAL		= 1 << 1,
 	FTRACE_OPS_FL_DYNAMIC		= 1 << 2,
+	FTRACE_OPS_FL_CONTROL		= 1 << 3,
 };
 
 struct ftrace_ops {
 	ftrace_func_t			func;
 	struct ftrace_ops		*next;
 	unsigned long			flags;
+	int __percpu			*disabled;
 #ifdef CONFIG_DYNAMIC_FTRACE
 	struct ftrace_hash		*notrace_hash;
 	struct ftrace_hash		*filter_hash;
@@ -97,6 +113,56 @@ int register_ftrace_function(struct ftrace_ops *ops);
 int unregister_ftrace_function(struct ftrace_ops *ops);
 void clear_ftrace_function(void);
 
+/**
+ * ftrace_function_local_enable - enable controlled ftrace_ops on current cpu
+ *
+ * This function enables tracing on current cpu by decreasing
+ * the per cpu control variable.
+ * It must be called with preemption disabled and only on
+ * ftrace_ops registered with FTRACE_OPS_FL_CONTROL.
+ */
+static inline void ftrace_function_local_enable(struct ftrace_ops *ops)
+{
+	if (WARN_ON_ONCE(!(ops->flags & FTRACE_OPS_FL_CONTROL) ||
+			 !preempt_count()))
+		return;
+
+	(*this_cpu_ptr(ops->disabled))--;
+}
+
+/**
+ * ftrace_function_local_disable - enable controlled ftrace_ops on current cpu
+ *
+ * This function enables tracing on current cpu by decreasing
+ * the per cpu control variable.
+ * It must be called with preemption disabled and only on
+ * ftrace_ops registered with FTRACE_OPS_FL_CONTROL.
+ */
+static inline void ftrace_function_local_disable(struct ftrace_ops *ops)
+{
+	if (WARN_ON_ONCE(!(ops->flags & FTRACE_OPS_FL_CONTROL) ||
+			 !preempt_count()))
+		return;
+
+	(*this_cpu_ptr(ops->disabled))++;
+}
+
+/**
+ * ftrace_function_local_disabled - returns ftrace_ops disabled value
+ *                                  on current cpu
+ *
+ * This function returns value of ftrace_ops::disabled on current cpu.
+ * It must be called with preemption disabled and only on
+ * ftrace_ops registered with FTRACE_OPS_FL_CONTROL.
+ */
+static inline int ftrace_function_local_disabled(struct ftrace_ops *ops)
+{
+	WARN_ON_ONCE(!(ops->flags & FTRACE_OPS_FL_CONTROL) ||
+		     !preempt_count());
+
+	return *this_cpu_ptr(ops->disabled);
+}
+
 extern void ftrace_stub(unsigned long a0, unsigned long a1);
 
 #else /* !CONFIG_FUNCTION_TRACER */
kernel/trace/ftrace.c +104 -7

166 lines not on this page · raw message

kernel/trace/trace.h +2 -0
index b93ecba..55c6ea0 100644
@@ -288,6 +288,8 @@ struct tracer {
 /* for function tracing recursion */
 #define TRACE_INTERNAL_BIT		(1<<11)
 #define TRACE_GLOBAL_BIT		(1<<12)
+#define TRACE_CONTROL_BIT		(1<<13)
+
 /*
  * Abuse of the trace_recursion.
  * As we need a way to maintain state if we are tracing the function
author
#124 lore raw

Re: [PATCH 2/7] ftrace: Add enable/disable ftrace_ops control interface

Quoted 40 lines · Jiri Olsa
Quoted 26 lines · Frederic Weisbecker
Quoted 25 lines · Steven Rostedt
Quoted 23 lines · Frederic Weisbecker
Quoted 10 lines
include/linux/ftrace.h +9 -0
+/**
+ * ftrace_function_enable - enable controlled ftrace_ops on given cpu
+ *
+ * This function enables tracing on given cpu by decreasing
+ * the per cpu control variable.
+ * It must be called with preemption disabled and only on
+ * ftrace_ops registered with FTRACE_OPS_FL_CONTROL.
+ */
+static inline void ftrace_function_enable(struct ftrace_ops *ops, int cpu)

include/linux/ftrace.h int register_ftrace_function(struct ftrace_ops *ops); · line 148

Quoted 10 lines
include/linux/ftrace.h +10 -0
+{
+	atomic_t *disabled;
+
+	if (WARN_ON_ONCE(!(ops->flags & FTRACE_OPS_FL_CONTROL) ||
+			 !preempt_count()))
+		return;
+
+	disabled = per_cpu_ptr(ops->disabled, cpu);
+	atomic_dec(disabled);
+}

As you're using this for the local CPU exclusively, I suggest you rather rename it to "ftrace_function_{dis,en}able_cpu(struct ftrace_ops *ops)"

Quoted 7 lines
Quoted 2 lines

I wonder if "ftrace_function_local_{dis,en}able(ops)" would be better? That would match something like local_irq_disable/enable.

Good idea.

Quoted 2 lines
Quoted 2 lines

and use __get_cpu_var() that does the preempt check for you.

I haven't found preempt check this_cpu_ptr path.. not sure I missed it.. so I'm keeping the implicit preemt check.
Attaching new version of the patch (I'll need to resend whole serie once this one is agreed).

Quoted 6 lines

changes:
- using this_cpu_ptr to touch ftrace_ops::disable
- renamed ftrace_ops:disable API:
void ftrace_function_local_enable(struct ftrace_ops *ops) void ftrace_function_local_disable(struct ftrace_ops *ops) int ftrace_function_local_disabled(struct ftrace_ops *ops)

ops, missed one... using int instead of atomic_t for ftrace_ops::disable

jirka

#125 lore raw

Re: [PATCH 2/7] ftrace: Add enable/disable ftrace_ops control interface

In reply to Jiri Olsa
Quoted 30 lines · Jiri Olsa
Quoted 29 lines · Frederic Weisbecker
Quoted 27 lines · Steven Rostedt
Quoted 15 lines · Frederic Weisbecker
Quoted 15 lines
include/linux/ftrace.h +14 -0
+/**
+ * ftrace_function_enable - enable controlled ftrace_ops on given cpu
+ *
+ * This function enables tracing on given cpu by decreasing
+ * the per cpu control variable.
+ * It must be called with preemption disabled and only on
+ * ftrace_ops registered with FTRACE_OPS_FL_CONTROL.
+ */
+static inline void ftrace_function_enable(struct ftrace_ops *ops, int cpu)
+{
+	atomic_t *disabled;
+
+	if (WARN_ON_ONCE(!(ops->flags & FTRACE_OPS_FL_CONTROL) ||
+			 !preempt_count()))
Quoted 8 lines

include/linux/ftrace.h int register_ftrace_function(struct ftrace_ops *ops); · line 131

Quoted 5 lines
include/linux/ftrace.h +5 -0
+		return;
+
+	disabled = per_cpu_ptr(ops->disabled, cpu);
+	atomic_dec(disabled);
+}

As you're using this for the local CPU exclusively, I suggest you rather rename it to "ftrace_function_{dis,en}able_cpu(struct ftrace_ops *ops)"

I wonder if "ftrace_function_local_{dis,en}able(ops)" would be better? That would match something like local_irq_disable/enable.

Quoted 7 lines
Quoted 4 lines

Good idea.

Quoted 2 lines
Quoted 2 lines

and use __get_cpu_var() that does the preempt check for you.

I haven't found preempt check this_cpu_ptr path.. not sure I missed it.. so I'm keeping the implicit preemt check.

#ifdef CONFIG_DEBUG_PREEMPT
#define my_cpu_offset per_cpu_offset(smp_processor_id()) #else
#define my_cpu_offset __my_cpu_offset
#endif

#ifdef CONFIG_DEBUG_PREEMPT
#define this_cpu_ptr(ptr) SHIFT_PERCPU_PTR(ptr, my_cpu_offset) #else
#define this_cpu_ptr(ptr) __this_cpu_ptr(ptr)
#endif

And smp_processor_id() has a preemption check.

author
#126 lore raw

Re: [PATCH 2/7] ftrace: Add enable/disable ftrace_ops control interface

Quoted 45 lines · Frederic Weisbecker
Quoted 30 lines · Jiri Olsa
Quoted 29 lines · Frederic Weisbecker
Quoted 27 lines · Steven Rostedt
Quoted 15 lines · Frederic Weisbecker
Quoted 15 lines
include/linux/ftrace.h +14 -0
+/**
+ * ftrace_function_enable - enable controlled ftrace_ops on given cpu
+ *
+ * This function enables tracing on given cpu by decreasing
+ * the per cpu control variable.
+ * It must be called with preemption disabled and only on
+ * ftrace_ops registered with FTRACE_OPS_FL_CONTROL.
+ */
+static inline void ftrace_function_enable(struct ftrace_ops *ops, int cpu)
+{
+	atomic_t *disabled;
+
+	if (WARN_ON_ONCE(!(ops->flags & FTRACE_OPS_FL_CONTROL) ||
+			 !preempt_count()))
Quoted 8 lines

include/linux/ftrace.h int register_ftrace_function(struct ftrace_ops *ops); · line 131

Quoted 5 lines
include/linux/ftrace.h +5 -0
+		return;
+
+	disabled = per_cpu_ptr(ops->disabled, cpu);
+	atomic_dec(disabled);
+}

As you're using this for the local CPU exclusively, I suggest you rather rename it to "ftrace_function_{dis,en}able_cpu(struct ftrace_ops *ops)"

I wonder if "ftrace_function_local_{dis,en}able(ops)" would be better? That would match something like local_irq_disable/enable.

Quoted 7 lines
Quoted 4 lines

Good idea.

Quoted 2 lines
Quoted 2 lines

and use __get_cpu_var() that does the preempt check for you.

I haven't found preempt check this_cpu_ptr path.. not sure I missed it.. so I'm keeping the implicit preemt check.

#ifdef CONFIG_DEBUG_PREEMPT
#define my_cpu_offset per_cpu_offset(smp_processor_id()) #else
#define my_cpu_offset __my_cpu_offset
#endif

Quoted 7 lines

#ifdef CONFIG_DEBUG_PREEMPT
#define this_cpu_ptr(ptr) SHIFT_PERCPU_PTR(ptr, my_cpu_offset) #else
#define this_cpu_ptr(ptr) __this_cpu_ptr(ptr)
#endif

And smp_processor_id() has a preemption check.

yay.. ok :) so this one is triggered only if there's CONFIG_DEBUG_PREEMPT option enabled.. seems to me it'd better to keep the implicit check anyway.

jirka

#127 lore raw

Re: [PATCH 2/7] ftrace: Add enable/disable ftrace_ops control interface

Quoted 52 lines · Jiri Olsa
Quoted 51 lines · Frederic Weisbecker
Quoted 30 lines · Jiri Olsa
Quoted 29 lines · Frederic Weisbecker
Quoted 27 lines · Steven Rostedt
Quoted 15 lines · Frederic Weisbecker
Quoted 15 lines
include/linux/ftrace.h +14 -0
+/**
+ * ftrace_function_enable - enable controlled ftrace_ops on given cpu
+ *
+ * This function enables tracing on given cpu by decreasing
+ * the per cpu control variable.
+ * It must be called with preemption disabled and only on
+ * ftrace_ops registered with FTRACE_OPS_FL_CONTROL.
+ */
+static inline void ftrace_function_enable(struct ftrace_ops *ops, int cpu)
+{
+	atomic_t *disabled;
+
+	if (WARN_ON_ONCE(!(ops->flags & FTRACE_OPS_FL_CONTROL) ||
+			 !preempt_count()))
Quoted 8 lines

include/linux/ftrace.h int register_ftrace_function(struct ftrace_ops *ops); · line 131

Quoted 5 lines
include/linux/ftrace.h +5 -0
+		return;
+
+	disabled = per_cpu_ptr(ops->disabled, cpu);
+	atomic_dec(disabled);
+}

As you're using this for the local CPU exclusively, I suggest you rather rename it to "ftrace_function_{dis,en}able_cpu(struct ftrace_ops *ops)"

I wonder if "ftrace_function_local_{dis,en}able(ops)" would be better? That would match something like local_irq_disable/enable.

Quoted 7 lines
Quoted 4 lines

Good idea.

Quoted 2 lines
Quoted 2 lines

and use __get_cpu_var() that does the preempt check for you.

I haven't found preempt check this_cpu_ptr path.. not sure I missed it.. so I'm keeping the implicit preemt check.

#ifdef CONFIG_DEBUG_PREEMPT
#define my_cpu_offset per_cpu_offset(smp_processor_id()) #else
#define my_cpu_offset __my_cpu_offset
#endif

#ifdef CONFIG_DEBUG_PREEMPT
#define this_cpu_ptr(ptr) SHIFT_PERCPU_PTR(ptr, my_cpu_offset) #else
#define this_cpu_ptr(ptr) __this_cpu_ptr(ptr)
#endif

Quoted 6 lines
Quoted 1 line

And smp_processor_id() has a preemption check.

yay.. ok :) so this one is triggered only if there's CONFIG_DEBUG_PREEMPT option enabled.. seems to me it'd better to keep the implicit check anyway.

jirka

This is a debugging option deemed to lower runtime debugging checks in production.

Is there a good reason to keep the check on every case?

author
#128 lore raw

Re: [PATCH 2/7] ftrace: Add enable/disable ftrace_ops control interface

Quoted 58 lines · Frederic Weisbecker
Quoted 57 lines · Jiri Olsa
Quoted 45 lines · Frederic Weisbecker
Quoted 30 lines · Jiri Olsa
Quoted 29 lines · Frederic Weisbecker
Quoted 27 lines · Steven Rostedt
Quoted 15 lines · Frederic Weisbecker
Quoted 15 lines
(quoted diff) +14 -0
+/**
+ * ftrace_function_enable - enable controlled ftrace_ops on given cpu
+ *
+ * This function enables tracing on given cpu by decreasing
+ * the per cpu control variable.
+ * It must be called with preemption disabled and only on
+ * ftrace_ops registered with FTRACE_OPS_FL_CONTROL.
+ */
+static inline void ftrace_function_enable(struct ftrace_ops *ops, int cpu)
+{
+	atomic_t *disabled;
+
+	if (WARN_ON_ONCE(!(ops->flags & FTRACE_OPS_FL_CONTROL) ||
+			 !preempt_count()))
Quoted 8 lines

Review

Quoted 5 lines
(quoted diff) +5 -0
+		return;
+
+	disabled = per_cpu_ptr(ops->disabled, cpu);
+	atomic_dec(disabled);
+}

As you're using this for the local CPU exclusively, I suggest you rather rename it to "ftrace_function_{dis,en}able_cpu(struct ftrace_ops *ops)"

I wonder if "ftrace_function_local_{dis,en}able(ops)" would be better? That would match something like local_irq_disable/enable.

Quoted 7 lines
Quoted 4 lines

Good idea.

Quoted 2 lines
Quoted 2 lines

and use __get_cpu_var() that does the preempt check for you.

I haven't found preempt check this_cpu_ptr path.. not sure I missed it.. so I'm keeping the implicit preemt check.

#ifdef CONFIG_DEBUG_PREEMPT
#define my_cpu_offset per_cpu_offset(smp_processor_id()) #else
#define my_cpu_offset __my_cpu_offset
#endif

Quoted 7 lines

#ifdef CONFIG_DEBUG_PREEMPT
#define this_cpu_ptr(ptr) SHIFT_PERCPU_PTR(ptr, my_cpu_offset) #else
#define this_cpu_ptr(ptr) __this_cpu_ptr(ptr)
#endif

And smp_processor_id() has a preemption check.

yay.. ok :) so this one is triggered only if there's CONFIG_DEBUG_PREEMPT option enabled.. seems to me it'd better to keep the implicit check anyway.

Quoted 7 lines
Quoted 1 line

jirka

This is a debugging option deemed to lower runtime debugging checks in production.

Is there a good reason to keep the check on every case?

none I guess, apart from me feeling better.. ;)
attached new version without the preemt_count check int the WARN_ON_ONCE

thanks,
jirka

---

include/linux/ftrace.h +62 -0
index f33fb3b..d95df4b 100644
@@ -31,16 +31,32 @@ ftrace_enable_sysctl(struct ctl_table *table, int write,
 
 typedef void (*ftrace_func_t)(unsigned long ip, unsigned long parent_ip);
 
+/*
+ * FTRACE_OPS_FL_* bits denote the state of ftrace_ops struct and are
+ * set in the flags member.
+ *
+ * ENABLED - set/unset when ftrace_ops is registered/unregistered
+ * GLOBAL  - set manualy by ftrace_ops user to denote the ftrace_ops
+ *           is part of the global tracers sharing the same filter
+ *           via set_ftrace_* debugfs files.
+ * DYNAMIC - set when ftrace_ops is registered to denote dynamically
+ *           allocated ftrace_ops which need special care
+ * CONTROL - set manualy by ftrace_ops user to denote the ftrace_ops
+ *           could be controled by following calls:
+ *           ftrace_function_enable, ftrace_function_disable
+ */
 enum {
 	FTRACE_OPS_FL_ENABLED		= 1 << 0,
 	FTRACE_OPS_FL_GLOBAL		= 1 << 1,
 	FTRACE_OPS_FL_DYNAMIC		= 1 << 2,
+	FTRACE_OPS_FL_CONTROL		= 1 << 3,
 };
 
 struct ftrace_ops {
 	ftrace_func_t			func;
 	struct ftrace_ops		*next;
 	unsigned long			flags;
+	int __percpu			*disabled;
 #ifdef CONFIG_DYNAMIC_FTRACE
 	struct ftrace_hash		*notrace_hash;
 	struct ftrace_hash		*filter_hash;
@@ -97,6 +113,52 @@ int register_ftrace_function(struct ftrace_ops *ops);
 int unregister_ftrace_function(struct ftrace_ops *ops);
 void clear_ftrace_function(void);
 
+/**
+ * ftrace_function_local_enable - enable controlled ftrace_ops on current cpu
+ *
+ * This function enables tracing on current cpu by decreasing
+ * the per cpu control variable.
+ * It must be called with preemption disabled and only on
+ * ftrace_ops registered with FTRACE_OPS_FL_CONTROL.
+ */
+static inline void ftrace_function_local_enable(struct ftrace_ops *ops)
+{
+	if (WARN_ON_ONCE(!(ops->flags & FTRACE_OPS_FL_CONTROL)))
+		return;
+
+	(*this_cpu_ptr(ops->disabled))--;
+}
+
+/**
+ * ftrace_function_local_disable - enable controlled ftrace_ops on current cpu
+ *
+ * This function enables tracing on current cpu by decreasing
+ * the per cpu control variable.
+ * It must be called with preemption disabled and only on
+ * ftrace_ops registered with FTRACE_OPS_FL_CONTROL.
+ */
+static inline void ftrace_function_local_disable(struct ftrace_ops *ops)
+{
+	if (WARN_ON_ONCE(!(ops->flags & FTRACE_OPS_FL_CONTROL)))
+		return;
+
+	(*this_cpu_ptr(ops->disabled))++;
+}
+
+/**
+ * ftrace_function_local_disabled - returns ftrace_ops disabled value
+ *                                  on current cpu
+ *
+ * This function returns value of ftrace_ops::disabled on current cpu.
+ * It must be called with preemption disabled and only on
+ * ftrace_ops registered with FTRACE_OPS_FL_CONTROL.
+ */
+static inline int ftrace_function_local_disabled(struct ftrace_ops *ops)
+{
+	WARN_ON_ONCE(!(ops->flags & FTRACE_OPS_FL_CONTROL));
+	return *this_cpu_ptr(ops->disabled);
+}
+
 extern void ftrace_stub(unsigned long a0, unsigned long a1);
 
 #else /* !CONFIG_FUNCTION_TRACER */
kernel/trace/ftrace.c +104 -7

166 lines not on this page · raw message

kernel/trace/trace.h +2 -0
index b93ecba..55c6ea0 100644
@@ -288,6 +288,8 @@ struct tracer {
 /* for function tracing recursion */
 #define TRACE_INTERNAL_BIT		(1<<11)
 #define TRACE_GLOBAL_BIT		(1<<12)
+#define TRACE_CONTROL_BIT		(1<<13)
+
 /*
  * Abuse of the trace_recursion.
  * As we need a way to maintain state if we are tracing the function
#129 lore raw

Re: [PATCH 2/7] ftrace: Add enable/disable ftrace_ops control interface

Quoted 62 lines · Jiri Olsa
Quoted 61 lines · Frederic Weisbecker
Quoted 58 lines · Jiri Olsa
Quoted 45 lines · Frederic Weisbecker
Quoted 30 lines · Jiri Olsa
Quoted 29 lines · Frederic Weisbecker
Quoted 27 lines · Steven Rostedt
Quoted 15 lines · Frederic Weisbecker
Quoted 15 lines
include/linux/ftrace.h +14 -0
+/**
+ * ftrace_function_enable - enable controlled ftrace_ops on given cpu
+ *
+ * This function enables tracing on given cpu by decreasing
+ * the per cpu control variable.
+ * It must be called with preemption disabled and only on
+ * ftrace_ops registered with FTRACE_OPS_FL_CONTROL.
+ */
+static inline void ftrace_function_enable(struct ftrace_ops *ops, int cpu)
+{
+	atomic_t *disabled;
+
+	if (WARN_ON_ONCE(!(ops->flags & FTRACE_OPS_FL_CONTROL) ||
+			 !preempt_count()))
Quoted 8 lines

include/linux/ftrace.h int register_ftrace_function(struct ftrace_ops *ops); · line 130

Quoted 5 lines
include/linux/ftrace.h +5 -0
+		return;
+
+	disabled = per_cpu_ptr(ops->disabled, cpu);
+	atomic_dec(disabled);
+}

As you're using this for the local CPU exclusively, I suggest you rather rename it to "ftrace_function_{dis,en}able_cpu(struct ftrace_ops *ops)"

I wonder if "ftrace_function_local_{dis,en}able(ops)" would be better? That would match something like local_irq_disable/enable.

Quoted 7 lines
Quoted 4 lines

Good idea.

Quoted 2 lines
Quoted 2 lines

and use __get_cpu_var() that does the preempt check for you.

I haven't found preempt check this_cpu_ptr path.. not sure I missed it.. so I'm keeping the implicit preemt check.

#ifdef CONFIG_DEBUG_PREEMPT
#define my_cpu_offset per_cpu_offset(smp_processor_id()) #else
#define my_cpu_offset __my_cpu_offset
#endif

Quoted 7 lines

#ifdef CONFIG_DEBUG_PREEMPT
#define this_cpu_ptr(ptr) SHIFT_PERCPU_PTR(ptr, my_cpu_offset) #else
#define this_cpu_ptr(ptr) __this_cpu_ptr(ptr)
#endif

And smp_processor_id() has a preemption check.

yay.. ok :) so this one is triggered only if there's CONFIG_DEBUG_PREEMPT option enabled.. seems to me it'd better to keep the implicit check anyway.

jirka

Quoted 7 lines
Quoted 4 lines

This is a debugging option deemed to lower runtime debugging checks in production.

Is there a good reason to keep the check on every case?

none I guess, apart from me feeling better.. ;)
attached new version without the preemt_count check int the WARN_ON_ONCE

Looks good.

Thanks.

maintainer
#130 lore raw

Re: [PATCH 2/7] ftrace: Add enable/disable ftrace_ops control interface

In reply to Jiri Olsa
Quoted 2 lines · Jiri Olsa

yay.. ok :) so this one is triggered only if there's CONFIG_DEBUG_PREEMPT option enabled.. seems to me it'd better to keep the implicit check anyway.

If the per_cpu pointer usage already warns if preemption is not disabled, then we don't need the extra check. I think I was the one to recommend adding it, but if the warning is already there, I don't think it is necessary. You can still keep a comment, and even say, the per_cpu pointer usage will complain when CONFIG_DEBUG_PREEMPT is enabled, if this is called without preemption disabled.

-- Steve

author
#131 lore raw

PATCH v5 3/7 ftrace, perf: Add open/close tracepoint perf registration actions

In reply to Jiri Olsa

Adding TRACE_REG_PERF_OPEN and TRACE_REG_PERF_CLOSE to differentiate register/unregister from open/close actions.

The register/unregister actions are invoked for the first/last tracepoint user when opening/closing the evet.

The open/close actions are invoked for each tracepoint user when opening/closing the event.

---
 include/linux/ftrace_event.h    |    6 +-
 kernel/trace/trace.h            |    5 ++
 kernel/trace/trace_event_perf.c |  116 +++++++++++++++++++++++++--------------
 kernel/trace/trace_events.c     |   10 ++-
 kernel/trace/trace_kprobe.c     |    6 ++-
 kernel/trace/trace_syscalls.c   |   14 +++-
 6 files changed, 106 insertions(+), 51 deletions(-)
include/linux/ftrace_event.h +4 -2
index c3da42d..195e360 100644
@@ -146,6 +146,8 @@ enum trace_reg {
 	TRACE_REG_UNREGISTER,
 	TRACE_REG_PERF_REGISTER,
 	TRACE_REG_PERF_UNREGISTER,
+	TRACE_REG_PERF_OPEN,
+	TRACE_REG_PERF_CLOSE,
 };
 
 struct ftrace_event_call;
@@ -157,7 +159,7 @@ struct ftrace_event_class {
 	void			*perf_probe;
 #endif
 	int			(*reg)(struct ftrace_event_call *event,
-				       enum trace_reg type);
+				       enum trace_reg type, void *data);
 	int			(*define_fields)(struct ftrace_event_call *);
 	struct list_head	*(*get_fields)(struct ftrace_event_call *);
 	struct list_head	fields;
@@ -165,7 +167,7 @@ struct ftrace_event_class {
 };
 
 extern int ftrace_event_reg(struct ftrace_event_call *event,
-			    enum trace_reg type);
+			    enum trace_reg type, void *data);
 
 enum {
 	TRACE_EVENT_FL_ENABLED_BIT,
kernel/trace/trace.h +5 -0
index 41c54e3..85732a8 100644
@@ -828,4 +828,9 @@ extern const char *__stop___trace_bprintk_fmt[];
 	FTRACE_ENTRY(call, struct_name, id, PARAMS(tstruct), PARAMS(print))
 #include "trace_entries.h"
 
+#ifdef CONFIG_PERF_EVENTS
+int perf_ftrace_event_register(struct ftrace_event_call *call,
+			       enum trace_reg type, void *data);
+#endif
+
 #endif /* _LINUX_KERNEL_TRACE_H */
kernel/trace/trace_event_perf.c +75 -41

154 lines not on this page · raw message

kernel/trace/trace_events.c +7 -3
index c212a7f..5138fea 100644
@@ -147,7 +147,8 @@ int trace_event_raw_init(struct ftrace_event_call *call)
 }
 EXPORT_SYMBOL_GPL(trace_event_raw_init);
 
-int ftrace_event_reg(struct ftrace_event_call *call, enum trace_reg type)
+int ftrace_event_reg(struct ftrace_event_call *call,
+		     enum trace_reg type, void *data)
 {
 	switch (type) {
 	case TRACE_REG_REGISTER:
@@ -170,6 +171,9 @@ int ftrace_event_reg(struct ftrace_event_call *call, enum trace_reg type)
 					    call->class->perf_probe,
 					    call);
 		return 0;
+	case TRACE_REG_PERF_OPEN:
+	case TRACE_REG_PERF_CLOSE:
+		return 0;
 #endif
 	}
 	return 0;
@@ -209,7 +213,7 @@ static int ftrace_event_enable_disable(struct ftrace_event_call *call,
 				tracing_stop_cmdline_record();
 				call->flags &= ~TRACE_EVENT_FL_RECORDED_CMD;
 			}
-			call->class->reg(call, TRACE_REG_UNREGISTER);
+			call->class->reg(call, TRACE_REG_UNREGISTER, NULL);
 		}
 		break;
 	case 1:
@@ -218,7 +222,7 @@ static int ftrace_event_enable_disable(struct ftrace_event_call *call,
 				tracing_start_cmdline_record();
 				call->flags |= TRACE_EVENT_FL_RECORDED_CMD;
 			}
-			ret = call->class->reg(call, TRACE_REG_REGISTER);
+			ret = call->class->reg(call, TRACE_REG_REGISTER, NULL);
 			if (ret) {
 				tracing_stop_cmdline_record();
 				pr_info("event trace: Could not enable event "
kernel/trace/trace_kprobe.c +5 -1
index 00d527c..5667f89 100644
@@ -1892,7 +1892,8 @@ static __kprobes void kretprobe_perf_func(struct kretprobe_instance *ri,
 #endif	/* CONFIG_PERF_EVENTS */
 
 static __kprobes
-int kprobe_register(struct ftrace_event_call *event, enum trace_reg type)
+int kprobe_register(struct ftrace_event_call *event,
+		    enum trace_reg type, void *data)
 {
 	struct trace_probe *tp = (struct trace_probe *)event->data;
 
@@ -1909,6 +1910,9 @@ int kprobe_register(struct ftrace_event_call *event, enum trace_reg type)
 	case TRACE_REG_PERF_UNREGISTER:
 		disable_trace_probe(tp, TP_FLAG_PROFILE);
 		return 0;
+	case TRACE_REG_PERF_OPEN:
+	case TRACE_REG_PERF_CLOSE:
+		return 0;
 #endif
 	}
 	return 0;
kernel/trace/trace_syscalls.c +10 -4
index cb65454..6916b0d 100644
@@ -17,9 +17,9 @@ static DECLARE_BITMAP(enabled_enter_syscalls, NR_syscalls);
 static DECLARE_BITMAP(enabled_exit_syscalls, NR_syscalls);
 
 static int syscall_enter_register(struct ftrace_event_call *event,
-				 enum trace_reg type);
+				 enum trace_reg type, void *data);
 static int syscall_exit_register(struct ftrace_event_call *event,
-				 enum trace_reg type);
+				 enum trace_reg type, void *data);
 
 static int syscall_enter_define_fields(struct ftrace_event_call *call);
 static int syscall_exit_define_fields(struct ftrace_event_call *call);
@@ -649,7 +649,7 @@ void perf_sysexit_disable(struct ftrace_event_call *call)
 #endif /* CONFIG_PERF_EVENTS */
 
 static int syscall_enter_register(struct ftrace_event_call *event,
-				 enum trace_reg type)
+				 enum trace_reg type, void *data)
 {
 	switch (type) {
 	case TRACE_REG_REGISTER:
@@ -664,13 +664,16 @@ static int syscall_enter_register(struct ftrace_event_call *event,
 	case TRACE_REG_PERF_UNREGISTER:
 		perf_sysenter_disable(event);
 		return 0;
+	case TRACE_REG_PERF_OPEN:
+	case TRACE_REG_PERF_CLOSE:
+		return 0;
 #endif
 	}
 	return 0;
 }
 
 static int syscall_exit_register(struct ftrace_event_call *event,
-				 enum trace_reg type)
+				 enum trace_reg type, void *data)
 {
 	switch (type) {
 	case TRACE_REG_REGISTER:
@@ -685,6 +688,9 @@ static int syscall_exit_register(struct ftrace_event_call *event,
 	case TRACE_REG_PERF_UNREGISTER:
 		perf_sysexit_disable(event);
 		return 0;
+	case TRACE_REG_PERF_OPEN:
+	case TRACE_REG_PERF_CLOSE:
+		return 0;
 #endif
 	}
 	return 0;
Signature · 1 line
1.7.1
author
#132 lore raw

PATCH v5 4/7 ftrace, perf: Add add/del tracepoint perf registration actions

In reply to Jiri Olsa

Adding TRACE_REG_PERF_ADD and TRACE_REG_PERF_DEL to handle perf event schedule in/out actions.

The add action is invoked for when the perf event is scheduled in, while the del action is invoked when the event is scheduled out.

---
 include/linux/ftrace_event.h    |    2 ++
 kernel/trace/trace_event_perf.c |    4 +++-
 kernel/trace/trace_events.c     |    2 ++
 kernel/trace/trace_kprobe.c     |    2 ++
 kernel/trace/trace_syscalls.c   |    4 ++++
 5 files changed, 13 insertions(+), 1 deletions(-)
include/linux/ftrace_event.h +2 -0
index 195e360..2bf677c 100644
@@ -148,6 +148,8 @@ enum trace_reg {
 	TRACE_REG_PERF_UNREGISTER,
 	TRACE_REG_PERF_OPEN,
 	TRACE_REG_PERF_CLOSE,
+	TRACE_REG_PERF_ADD,
+	TRACE_REG_PERF_DEL,
 };
 
 struct ftrace_event_call;
kernel/trace/trace_event_perf.c +3 -1
index 0cfcc37..d72af0b 100644
@@ -211,12 +211,14 @@ int perf_trace_add(struct perf_event *p_event, int flags)
 	list = this_cpu_ptr(pcpu_list);
 	hlist_add_head_rcu(&p_event->hlist_entry, list);
 
-	return 0;
+	return tp_event->class->reg(tp_event, TRACE_REG_PERF_ADD, p_event);
 }
 
 void perf_trace_del(struct perf_event *p_event, int flags)
 {
+	struct ftrace_event_call *tp_event = p_event->tp_event;
 	hlist_del_rcu(&p_event->hlist_entry);
+	tp_event->class->reg(tp_event, TRACE_REG_PERF_DEL, p_event);
 }
 
 __kprobes void *perf_trace_buf_prepare(int size, unsigned short type,
kernel/trace/trace_events.c +2 -0
index 5138fea..079a93a 100644
@@ -173,6 +173,8 @@ int ftrace_event_reg(struct ftrace_event_call *call,
 		return 0;
 	case TRACE_REG_PERF_OPEN:
 	case TRACE_REG_PERF_CLOSE:
+	case TRACE_REG_PERF_ADD:
+	case TRACE_REG_PERF_DEL:
 		return 0;
 #endif
 	}
kernel/trace/trace_kprobe.c +2 -0
index 5667f89..580a05e 100644
@@ -1912,6 +1912,8 @@ int kprobe_register(struct ftrace_event_call *event,
 		return 0;
 	case TRACE_REG_PERF_OPEN:
 	case TRACE_REG_PERF_CLOSE:
+	case TRACE_REG_PERF_ADD:
+	case TRACE_REG_PERF_DEL:
 		return 0;
 #endif
 	}
kernel/trace/trace_syscalls.c +4 -0
index 6916b0d..dbdd804 100644
@@ -666,6 +666,8 @@ static int syscall_enter_register(struct ftrace_event_call *event,
 		return 0;
 	case TRACE_REG_PERF_OPEN:
 	case TRACE_REG_PERF_CLOSE:
+	case TRACE_REG_PERF_ADD:
+	case TRACE_REG_PERF_DEL:
 		return 0;
 #endif
 	}
@@ -690,6 +692,8 @@ static int syscall_exit_register(struct ftrace_event_call *event,
 		return 0;
 	case TRACE_REG_PERF_OPEN:
 	case TRACE_REG_PERF_CLOSE:
+	case TRACE_REG_PERF_ADD:
+	case TRACE_REG_PERF_DEL:
 		return 0;
 #endif
 	}
Signature · 1 line
1.7.1
author
#133 lore raw

PATCH v5 5/7 ftrace, perf: Add support to use function tracepoint in perf

In reply to Jiri Olsa

Adding perf registration support for the ftrace function event, so it is now possible to register it via perf interface.

The perf_event struct statically contains ftrace_ops as a handle for function tracer. The function tracer is registered/unregistered in open/close actions.

To be efficient, we enable/disable ftrace_ops each time the traced process is scheduled in/out (via TRACE_REG_PERF_(ADD|DELL) handlers). This way tracing is enabled only when the process is running. Intentionally using this way instead of the event's hw state PERF_HES_STOPPED, which would not disable the ftrace_ops.

It is now possible to use function trace within perf commands like:

perf record -e ftrace:function ls
perf stat -e ftrace:function ls

Allowed only for root.

---
 include/linux/perf_event.h      |    3 +
 kernel/trace/trace.h            |    2 +
 kernel/trace/trace_event_perf.c |   92 +++++++++++++++++++++++++++++++++++++++
 kernel/trace/trace_export.c     |   28 ++++++++++++
 4 files changed, 125 insertions(+), 0 deletions(-)
include/linux/perf_event.h +3 -0
index 0b91db2..5003be6 100644
@@ -858,6 +858,9 @@ struct perf_event {
 #ifdef CONFIG_EVENT_TRACING
 	struct ftrace_event_call	*tp_event;
 	struct event_filter		*filter;
+#ifdef CONFIG_FUNCTION_TRACER
+	struct ftrace_ops               ftrace_ops;
+#endif
 #endif
 
 #ifdef CONFIG_CGROUP_PERF
kernel/trace/trace.h +2 -0
index 85732a8..e88e58a 100644
@@ -591,6 +591,8 @@ static inline int ftrace_trace_task(struct task_struct *task)
 static inline int ftrace_is_dead(void) { return 0; }
 #endif
 
+int ftrace_event_is_function(struct ftrace_event_call *call);
+
 /*
  * struct trace_parser - servers for reading the user input separated by spaces
  * @cont: set if the input is not complete - no final space char was found
kernel/trace/trace_event_perf.c +92 -0

101 lines not on this page · raw message

kernel/trace/trace_export.c +28 -0
index bbeec31..867653c 100644
@@ -131,6 +131,28 @@ ftrace_define_fields_##name(struct ftrace_event_call *event_call)	\
 
 #include "trace_entries.h"
 
+static int ftrace_event_class_register(struct ftrace_event_call *call,
+				       enum trace_reg type, void *data)
+{
+	switch (type) {
+	case TRACE_REG_PERF_REGISTER:
+	case TRACE_REG_PERF_UNREGISTER:
+		return 0;
+	case TRACE_REG_PERF_OPEN:
+	case TRACE_REG_PERF_CLOSE:
+	case TRACE_REG_PERF_ADD:
+	case TRACE_REG_PERF_DEL:
+#ifdef CONFIG_PERF_EVENTS
+		return perf_ftrace_event_register(call, type, data);
+#endif
+	case TRACE_REG_REGISTER:
+	case TRACE_REG_UNREGISTER:
+		break;
+	}
+
+	return -EINVAL;
+}
+
 #undef __entry
 #define __entry REC
 
@@ -159,6 +181,7 @@ struct ftrace_event_class event_class_ftrace_##call = {			\
 	.system			= __stringify(TRACE_SYSTEM),		\
 	.define_fields		= ftrace_define_fields_##call,		\
 	.fields			= LIST_HEAD_INIT(event_class_ftrace_##call.fields),\
+	.reg			= ftrace_event_class_register,		\
 };									\
 									\
 struct ftrace_event_call __used event_##call = {			\
@@ -170,4 +193,9 @@ struct ftrace_event_call __used event_##call = {			\
 struct ftrace_event_call __used						\
 __attribute__((section("_ftrace_events"))) *__event_##call = &event_##call;
 
+int ftrace_event_is_function(struct ftrace_event_call *call)
+{
+	return call == &event_function;
+}
+
 #include "trace_entries.h"
Signature · 1 line
1.7.1
author
#134 lore raw

PATCH v5 6/7 ftrace, perf: Distinguish ftrace function event field type

In reply to Jiri Olsa

Adding FILTER_TRACE_FN event field type for function tracepoint event, so it can be properly recognized within filtering code.

Currently all fields of ftrace subsystem events share the common field type FILTER_OTHER. Since the function trace fields need special care within the filtering code we need to recognize it properly, hence adding the FILTER_TRACE_FN event type.

---
 include/linux/ftrace_event.h       |    1 +
 kernel/trace/trace_events_filter.c |    7 ++++++-
 kernel/trace/trace_export.c        |   25 ++++++++++++++++++++-----
 3 files changed, 27 insertions(+), 6 deletions(-)
include/linux/ftrace_event.h +1 -0
index 2bf677c..dd478fc 100644
@@ -245,6 +245,7 @@ enum {
 	FILTER_STATIC_STRING,
 	FILTER_DYN_STRING,
 	FILTER_PTR_STRING,
+	FILTER_TRACE_FN,
 };
 
 #define EVENT_STORAGE_SIZE 128
kernel/trace/trace_events_filter.c +6 -1
index 24aee71..eb04a2a 100644
@@ -900,6 +900,11 @@ int filter_assign_type(const char *type)
 	return FILTER_OTHER;
 }
 
+static bool is_function_field(struct ftrace_event_field *field)
+{
+	return field->filter_type == FILTER_TRACE_FN;
+}
+
 static bool is_string_field(struct ftrace_event_field *field)
 {
 	return field->filter_type == FILTER_DYN_STRING ||
@@ -987,7 +992,7 @@ static int init_pred(struct filter_parse_state *ps,
 			fn = filter_pred_strloc;
 		else
 			fn = filter_pred_pchar;
-	} else {
+	} else if (!is_function_field(field)) {
 		if (field->is_signed)
 			ret = strict_strtoll(pred->regex.pattern, 0, &val);
 		else
kernel/trace/trace_export.c +20 -5
index 867653c..46c35e2 100644
@@ -67,7 +67,7 @@ static void __always_unused ____ftrace_check_##name(void)	\
 	ret = trace_define_field(event_call, #type, #item,		\
 				 offsetof(typeof(field), item),		\
 				 sizeof(field.item),			\
-				 is_signed_type(type), FILTER_OTHER);	\
+				 is_signed_type(type), filter_type);	\
 	if (ret)							\
 		return ret;
 
@@ -77,7 +77,7 @@ static void __always_unused ____ftrace_check_##name(void)	\
 				 offsetof(typeof(field),		\
 					  container.item),		\
 				 sizeof(field.container.item),		\
-				 is_signed_type(type), FILTER_OTHER);	\
+				 is_signed_type(type), filter_type);	\
 	if (ret)							\
 		return ret;
 
@@ -91,7 +91,7 @@ static void __always_unused ____ftrace_check_##name(void)	\
 		ret = trace_define_field(event_call, event_storage, #item, \
 				 offsetof(typeof(field), item),		\
 				 sizeof(field.item),			\
-				 is_signed_type(type), FILTER_OTHER);	\
+				 is_signed_type(type), filter_type);	\
 		mutex_unlock(&event_storage_mutex);			\
 		if (ret)						\
 			return ret;					\
@@ -104,7 +104,7 @@ static void __always_unused ____ftrace_check_##name(void)	\
 				 offsetof(typeof(field),		\
 					  container.item),		\
 				 sizeof(field.container.item),		\
-				 is_signed_type(type), FILTER_OTHER);	\
+				 is_signed_type(type), filter_type);	\
 	if (ret)							\
 		return ret;
 
@@ -112,10 +112,24 @@ static void __always_unused ____ftrace_check_##name(void)	\
 #define __dynamic_array(type, item)					\
 	ret = trace_define_field(event_call, #type, #item,		\
 				 offsetof(typeof(field), item),		\
-				 0, is_signed_type(type), FILTER_OTHER);\
+				 0, is_signed_type(type), filter_type);\
 	if (ret)							\
 		return ret;
 
+#define FILTER_TYPE_TRACE_FN           FILTER_TRACE_FN
+#define FILTER_TYPE_TRACE_GRAPH_ENT    FILTER_OTHER
+#define FILTER_TYPE_TRACE_GRAPH_RET    FILTER_OTHER
+#define FILTER_TYPE_TRACE_CTX          FILTER_OTHER
+#define FILTER_TYPE_TRACE_WAKE         FILTER_OTHER
+#define FILTER_TYPE_TRACE_STACK                FILTER_OTHER
+#define FILTER_TYPE_TRACE_USER_STACK   FILTER_OTHER
+#define FILTER_TYPE_TRACE_BPRINT       FILTER_OTHER
+#define FILTER_TYPE_TRACE_PRINT                FILTER_OTHER
+#define FILTER_TYPE_TRACE_MMIO_RW      FILTER_OTHER
+#define FILTER_TYPE_TRACE_MMIO_MAP     FILTER_OTHER
+#define FILTER_TYPE_TRACE_BRANCH       FILTER_OTHER
+#define FILTER_TYPE(arg)               FILTER_TYPE_##arg
+
 #undef FTRACE_ENTRY
 #define FTRACE_ENTRY(name, struct_name, id, tstruct, print)		\
 int									\
@@ -123,6 +137,7 @@ ftrace_define_fields_##name(struct ftrace_event_call *event_call)	\
 {									\
 	struct struct_name field;					\
 	int ret;							\
+	int filter_type = FILTER_TYPE(id);				\
 									\
 	tstruct;							\
 									\
Signature · 1 line
1.7.1
author
#135 lore raw

PATCH v5 7/7 ftrace, perf: Add filter support for function trace event

In reply to Jiri Olsa

Adding support to filter function trace event via perf interface. It is now possible to use filter interface in the perf tool like:

  perf record -e ftrace:function --filter="(ip == mm_*)" ls

The filter syntax is restricted to the the 'ip' field only, and following operators are accepted '==' '!=' '||', ending up with the filter strings like:

ip == f1[, ]f2 ... || ip != f3[, ]f4 ...

with comma ',' or space ' ' as a function separator. If the space ' ' is used as a separator, the right side of the assignment needs to be enclosed in double quotes '"'.

The '==' operator adds trace filter with same effect as would be added via set_ftrace_filter file.

The '!=' operator adds trace filter with same effect as would be added via set_ftrace_notrace file.

The right side of the '!=', '==' operators is list of functions or regexp. to be added to filter separated by space.

The '||' operator is used for connecting multiple filter definitions together. It is possible to have more than one '==' and '!=' operators within one filter string.

---
 include/linux/ftrace.h             |    1 +
 kernel/trace/ftrace.c              |    6 ++
 kernel/trace/trace.h               |    2 -
 kernel/trace/trace_event_perf.c    |    4 +-
 kernel/trace/trace_events_filter.c |  169 +++++++++++++++++++++++++++++++++---
 5 files changed, 168 insertions(+), 14 deletions(-)
include/linux/ftrace.h +1 -0
index d3f529c..60781ab 100644
@@ -240,6 +240,7 @@ int ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
 			int len, int reset);
 void ftrace_set_global_filter(unsigned char *buf, int len, int reset);
 void ftrace_set_global_notrace(unsigned char *buf, int len, int reset);
+void ftrace_free_filter(struct ftrace_ops *ops);
 
 int register_ftrace_command(struct ftrace_func_command *cmd);
 int unregister_ftrace_command(struct ftrace_func_command *cmd);
kernel/trace/ftrace.c +6 -0
index 45c9b0c..9935a2a 100644
@@ -1192,6 +1192,12 @@ static void free_ftrace_hash_rcu(struct ftrace_hash *hash)
 	call_rcu_sched(&hash->rcu, __free_ftrace_hash_rcu);
 }
 
+void ftrace_free_filter(struct ftrace_ops *ops)
+{
+	free_ftrace_hash(ops->filter_hash);
+	free_ftrace_hash(ops->notrace_hash);
+}
+
 static struct ftrace_hash *alloc_ftrace_hash(int size_bits)
 {
 	struct ftrace_hash *hash;
kernel/trace/trace.h +0 -2
index e88e58a..4ec6d18 100644
@@ -770,9 +770,7 @@ struct filter_pred {
 	u64 			val;
 	struct regex		regex;
 	unsigned short		*ops;
-#ifdef CONFIG_FTRACE_STARTUP_TEST
 	struct ftrace_event_field *field;
-#endif
 	int 			offset;
 	int 			not;
 	int 			op;
kernel/trace/trace_event_perf.c +3 -1
index 57eb232..220b50a 100644
@@ -298,7 +298,9 @@ static int perf_ftrace_function_register(struct perf_event *event)
 static int perf_ftrace_function_unregister(struct perf_event *event)
 {
 	struct ftrace_ops *ops = &event->ftrace_ops;
-	return unregister_ftrace_function(ops);
+	int ret = unregister_ftrace_function(ops);
+	ftrace_free_filter(ops);
+	return ret;
 }
 
 static void perf_ftrace_function_enable(struct perf_event *event)
kernel/trace/trace_events_filter.c +158 -11

239 lines not on this page · raw message

Signature · 1 line
1.7.1
maintainer
#136 lore raw
In reply to Jiri Olsa

Frederic,

Can you review and ACK this patch set (if you don't find anything wrong with it). I can then run it through a bunch of tests.

Thanks,

-- Steve

Quoted 63 lines · Jiri Olsa

hi,
here's another version of perf support for function trace with filter.

attached patches:
1/7 ftrace: Change filter/notrace set functions to return exit code 2/7 ftrace: Add enable/disable ftrace_ops control interface 3/7 ftrace, perf: Add open/close tracepoint perf registration actions 4/7 ftrace, perf: Add add/del tracepoint perf registration actions 5/7 ftrace, perf: Add support to use function tracepoint in perf 6/7 ftrace, perf: Distinguish ftrace function event field type 7/7 ftrace, perf: Add filter support for function trace event

v6 changes:
2/7 - added comments to FTRACE_OPS_FL_* bits enum
5/7 - added more info to the change log regarding ftrace_ops enable/disable 7/7 - rebased to the latest filter changes

v5 changes:
7/7 - fixed to properly support ',' in filter expressions

v4 changes:
2/7 - FL_GLOBAL_CONTROL changed to FL_GLOBAL_CONTROL_MASK
- changed WARN_ON_ONCE() to include the !preempt_count()
- changed this_cpu_ptr to per_cpu_ptr

ommited Fix possible NULL dereferencing in __ftrace_hash_rec_update (2/8 in v3)

v3 changes:
3/8 - renamed __add/remove_ftrace_ops
- fixed preemtp_enable/recursion_clear order in ftrace_ops_control_func
- renamed/commented API functions - enable/disable_ftrace_function

ommited graph tracer workarounf patch 10/10

v2 changes:
01/10 - keeping the old fix instead of adding hash_has_contents func
I'll send separating patchset for this
02/10 - using different way to avoid the issue (3/9 in v1) 03/10 - using the way proposed by Steven for controling ftrace_ops

         (4/9 in v1)
 06/10 - added check ensuring the ftrace:function event could be used by
         root only (7/9 in v1)

08/10 - added more description (8/9 in v1)
09/10 - changed '&&' operator to '||' which seems more suitable
in this case (9/9 in v1)

thanks,
jirka

---
 include/linux/ftrace.h             |   61 ++++++++++-
 include/linux/ftrace_event.h       |    9 +-
 include/linux/perf_event.h         |    3 +
 kernel/trace/ftrace.c              |  140 +++++++++++++++++++++--
 kernel/trace/trace.h               |   11 ++-
 kernel/trace/trace_event_perf.c    |  214 +++++++++++++++++++++++++++++-------
 kernel/trace/trace_events.c        |   12 ++-
 kernel/trace/trace_events_filter.c |  172 +++++++++++++++++++++++++++--
 kernel/trace/trace_export.c        |   53 ++++++++-
 kernel/trace/trace_kprobe.c        |    8 +-
 kernel/trace/trace_syscalls.c      |   18 +++-
 11 files changed, 617 insertions(+), 84 deletions(-)
author
#137 lore raw

v7 cover ftrace, perf: Adding support to use function trace

In reply to Jiri Olsa

hi,
here's another version of perf support for function trace with filter.

attached patches:
1/7 ftrace: Change filter/notrace set functions to return exit code 2/7 ftrace: Add enable/disable ftrace_ops control interface 3/7 ftrace, perf: Add open/close tracepoint perf registration actions 4/7 ftrace, perf: Add add/del tracepoint perf registration actions 5/7 ftrace, perf: Add support to use function tracepoint in perf 6/7 ftrace, perf: Distinguish ftrace function event field type 7/7 ftrace, perf: Add filter support for function trace event

v7 changes:
2/7 - using int instead of atomic_t for ftrace_ops::disable
- using this_cpu_ptr to touch ftrace_ops::disable
- renamed ftrace_ops:disable API
void ftrace_function_local_enable(struct ftrace_ops *ops) void ftrace_function_local_disable(struct ftrace_ops *ops) int ftrace_function_local_disabled(struct ftrace_ops *ops)

v6 changes:
2/7 - added comments to FTRACE_OPS_FL_* bits enum
5/7 - added more info to the change log regarding ftrace_ops enable/disable 7/7 - rebased to the latest filter changes

v5 changes:
7/7 - fixed to properly support ',' in filter expressions

v4 changes:
2/7 - FL_GLOBAL_CONTROL changed to FL_GLOBAL_CONTROL_MASK
- changed WARN_ON_ONCE() to include the !preempt_count()
- changed this_cpu_ptr to per_cpu_ptr

ommited Fix possible NULL dereferencing in __ftrace_hash_rec_update (2/8 in v3)

v3 changes:
3/8 - renamed __add/remove_ftrace_ops
- fixed preemtp_enable/recursion_clear order in ftrace_ops_control_func
- renamed/commented API functions - enable/disable_ftrace_function

ommited graph tracer workarounf patch 10/10

v2 changes:
01/10 - keeping the old fix instead of adding hash_has_contents func
I'll send separating patchset for this
02/10 - using different way to avoid the issue (3/9 in v1) 03/10 - using the way proposed by Steven for controling ftrace_ops

         (4/9 in v1)
 06/10 - added check ensuring the ftrace:function event could be used by
         root only (7/9 in v1)

08/10 - added more description (8/9 in v1)
09/10 - changed '&&' operator to '||' which seems more suitable
in this case (9/9 in v1)

thanks,
jirka

---
 include/linux/ftrace.h             |   70 ++++++++++++-
 include/linux/ftrace_event.h       |    9 +-
 include/linux/perf_event.h         |    3 +
 kernel/trace/ftrace.c              |  132 ++++++++++++++++++++--
 kernel/trace/trace.h               |   11 ++-
 kernel/trace/trace_event_perf.c    |  212 +++++++++++++++++++++++++++++-------
 kernel/trace/trace_events.c        |   12 ++-
 kernel/trace/trace_events_filter.c |  172 +++++++++++++++++++++++++++--
 kernel/trace/trace_export.c        |   53 ++++++++-
 kernel/trace/trace_kprobe.c        |    8 +-
 kernel/trace/trace_syscalls.c      |   18 +++-
 11 files changed, 616 insertions(+), 84 deletions(-)
author
#138 lore raw

PATCH v5 1/7 ftrace: Change filter/notrace set functions to return exit code

Currently the ftrace_set_filter and ftrace_set_notrace functions do not return any return code. So there's no way for ftrace_ops user to tell wether the filter was correctly applied.

The set_ftrace_filter interface returns error in case the filter did not match:

  # echo krava > set_ftrace_filter
  bash: echo: write error: Invalid argument

Changing both ftrace_set_filter and ftrace_set_notrace functions to return zero if the filter was applied correctly or -E* values in case of error.

---
 include/linux/ftrace.h |    4 ++--
 kernel/trace/ftrace.c  |   15 +++++++++------
 2 files changed, 11 insertions(+), 8 deletions(-)
include/linux/ftrace.h +2 -2
index 028e26f..f33fb3b 100644
@@ -178,9 +178,9 @@ struct dyn_ftrace {
 };
 
 int ftrace_force_update(void);
-void ftrace_set_filter(struct ftrace_ops *ops, unsigned char *buf,
+int ftrace_set_filter(struct ftrace_ops *ops, unsigned char *buf,
 		       int len, int reset);
-void ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
+int ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
 			int len, int reset);
 void ftrace_set_global_filter(unsigned char *buf, int len, int reset);
 void ftrace_set_global_notrace(unsigned char *buf, int len, int reset);
kernel/trace/ftrace.c +9 -6
index 683d559..e2e0597 100644
@@ -3146,8 +3146,10 @@ ftrace_set_regex(struct ftrace_ops *ops, unsigned char *buf, int len,
 	mutex_lock(&ftrace_regex_lock);
 	if (reset)
 		ftrace_filter_reset(hash);
-	if (buf)
-		ftrace_match_records(hash, buf, len);
+	if (buf && !ftrace_match_records(hash, buf, len)) {
+		ret = -EINVAL;
+		goto out_regex_unlock;
+	}
 
 	mutex_lock(&ftrace_lock);
 	ret = ftrace_hash_move(ops, enable, orig_hash, hash);
@@ -3157,6 +3159,7 @@ ftrace_set_regex(struct ftrace_ops *ops, unsigned char *buf, int len,
 
 	mutex_unlock(&ftrace_lock);
 
+ out_regex_unlock:
 	mutex_unlock(&ftrace_regex_lock);
 
 	free_ftrace_hash(hash);
@@ -3173,10 +3176,10 @@ ftrace_set_regex(struct ftrace_ops *ops, unsigned char *buf, int len,
  * Filters denote which functions should be enabled when tracing is enabled.
  * If @buf is NULL and reset is set, all functions will be enabled for tracing.
  */
-void ftrace_set_filter(struct ftrace_ops *ops, unsigned char *buf,
+int ftrace_set_filter(struct ftrace_ops *ops, unsigned char *buf,
 		       int len, int reset)
 {
-	ftrace_set_regex(ops, buf, len, reset, 1);
+	return ftrace_set_regex(ops, buf, len, reset, 1);
 }
 EXPORT_SYMBOL_GPL(ftrace_set_filter);
 
@@ -3191,10 +3194,10 @@ EXPORT_SYMBOL_GPL(ftrace_set_filter);
  * is enabled. If @buf is NULL and reset is set, all functions will be enabled
  * for tracing.
  */
-void ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
+int ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
 			int len, int reset)
 {
-	ftrace_set_regex(ops, buf, len, reset, 0);
+	return ftrace_set_regex(ops, buf, len, reset, 0);
 }
 EXPORT_SYMBOL_GPL(ftrace_set_notrace);
 /**
Signature · 1 line
1.7.1
Acked-by on v5 1/7
#139 lore raw

Re: [PATCH 1/7] ftrace: Change filter/notrace set functions to return exit code

Quoted 7 lines · Jiri Olsa

Currently the ftrace_set_filter and ftrace_set_notrace functions do not return any return code. So there's no way for ftrace_ops user to tell wether the filter was correctly applied.

The set_ftrace_filter interface returns error in case the filter did not match:

Quoted 8 lines
  # echo krava > set_ftrace_filter
  bash: echo: write error: Invalid argument

Changing both ftrace_set_filter and ftrace_set_notrace functions to return zero if the filter was applied correctly or -E* values in case of error.

author
#140 lore raw

PATCH v5 2/7 ftrace: Add enable/disable ftrace_ops control interface

In reply to Jiri Olsa

Adding a way to temporarily enable/disable ftrace_ops. The change follows the same way as 'global' ftrace_ops are done.

Introducing 2 global ftrace_ops - control_ops and ftrace_control_list which take over all ftrace_ops registered with FTRACE_OPS_FL_CONTROL flag. In addition new per cpu flag called 'disabled' is also added to ftrace_ops to provide the control information for each cpu.

When ftrace_ops with FTRACE_OPS_FL_CONTROL is registered, it is set as disabled for all cpus.

The ftrace_control_list contains all the registered 'control' ftrace_ops. The control_ops provides function which iterates ftrace_control_list and does the check for 'disabled' flag on current cpu.

Adding 3 inline functions:
ftrace_function_local_disable/ftrace_function_local_enable
- enable/disable the ftrace_ops on current cpu ftrace_function_local_disabled
- get disabled ftrace_ops::disabled value for current cpu

---
 include/linux/ftrace.h |   65 ++++++++++++++++++++++++++++
 kernel/trace/ftrace.c  |  111 +++++++++++++++++++++++++++++++++++++++++++++---
 kernel/trace/trace.h   |    2 +
 3 files changed, 171 insertions(+), 7 deletions(-)
include/linux/ftrace.h +65 -0
index f33fb3b..5cb3a51 100644
@@ -31,16 +31,32 @@ ftrace_enable_sysctl(struct ctl_table *table, int write,
 
 typedef void (*ftrace_func_t)(unsigned long ip, unsigned long parent_ip);
 
+/*
+ * FTRACE_OPS_FL_* bits denote the state of ftrace_ops struct and are
+ * set in the flags member.
+ *
+ * ENABLED - set/unset when ftrace_ops is registered/unregistered
+ * GLOBAL  - set manualy by ftrace_ops user to denote the ftrace_ops
+ *           is part of the global tracers sharing the same filter
+ *           via set_ftrace_* debugfs files.
+ * DYNAMIC - set when ftrace_ops is registered to denote dynamically
+ *           allocated ftrace_ops which need special care
+ * CONTROL - set manualy by ftrace_ops user to denote the ftrace_ops
+ *           could be controled by following calls:
+ *           ftrace_function_enable, ftrace_function_disable
+ */
 enum {
 	FTRACE_OPS_FL_ENABLED		= 1 << 0,
 	FTRACE_OPS_FL_GLOBAL		= 1 << 1,
 	FTRACE_OPS_FL_DYNAMIC		= 1 << 2,
+	FTRACE_OPS_FL_CONTROL		= 1 << 3,
 };
 
 struct ftrace_ops {
 	ftrace_func_t			func;
 	struct ftrace_ops		*next;
 	unsigned long			flags;
+	int __percpu			*disabled;
 #ifdef CONFIG_DYNAMIC_FTRACE
 	struct ftrace_hash		*notrace_hash;
 	struct ftrace_hash		*filter_hash;
@@ -97,6 +113,55 @@ int register_ftrace_function(struct ftrace_ops *ops);
 int unregister_ftrace_function(struct ftrace_ops *ops);
 void clear_ftrace_function(void);
 
+/**
+ * ftrace_function_local_enable - enable controlled ftrace_ops on current cpu
+ *
+ * This function enables tracing on current cpu by decreasing
+ * the per cpu control variable.
+ * It must be called with preemption disabled and only on ftrace_ops
+ * registered with FTRACE_OPS_FL_CONTROL. If called without preemption
+ * disabled, this_cpu_ptr will complain when CONFIG_DEBUG_PREEMPT is enabled.
+ */
+static inline void ftrace_function_local_enable(struct ftrace_ops *ops)
+{
+	if (WARN_ON_ONCE(!(ops->flags & FTRACE_OPS_FL_CONTROL)))
+		return;
+
+	(*this_cpu_ptr(ops->disabled))--;
+}
+
+/**
+ * ftrace_function_local_disable - enable controlled ftrace_ops on current cpu
+ *
+ * This function enables tracing on current cpu by decreasing
+ * the per cpu control variable.
+ * It must be called with preemption disabled and only on ftrace_ops
+ * registered with FTRACE_OPS_FL_CONTROL. If called without preemption
+ * disabled, this_cpu_ptr will complain when CONFIG_DEBUG_PREEMPT is enabled.
+ */
+static inline void ftrace_function_local_disable(struct ftrace_ops *ops)
+{
+	if (WARN_ON_ONCE(!(ops->flags & FTRACE_OPS_FL_CONTROL)))
+		return;
+
+	(*this_cpu_ptr(ops->disabled))++;
+}
+
+/**
+ * ftrace_function_local_disabled - returns ftrace_ops disabled value
+ *                                  on current cpu
+ *
+ * This function returns value of ftrace_ops::disabled on current cpu.
+ * It must be called with preemption disabled and only on ftrace_ops
+ * registered with FTRACE_OPS_FL_CONTROL. If called without preemption
+ * disabled, this_cpu_ptr will complain when CONFIG_DEBUG_PREEMPT is enabled.
+ */
+static inline int ftrace_function_local_disabled(struct ftrace_ops *ops)
+{
+	WARN_ON_ONCE(!(ops->flags & FTRACE_OPS_FL_CONTROL));
+	return *this_cpu_ptr(ops->disabled);
+}
+
 extern void ftrace_stub(unsigned long a0, unsigned long a1);
 
 #else /* !CONFIG_FUNCTION_TRACER */
kernel/trace/ftrace.c +104 -7

166 lines not on this page · raw message

kernel/trace/trace.h +2 -0
index b93ecba..55c6ea0 100644
@@ -288,6 +288,8 @@ struct tracer {
 /* for function tracing recursion */
 #define TRACE_INTERNAL_BIT		(1<<11)
 #define TRACE_GLOBAL_BIT		(1<<12)
+#define TRACE_CONTROL_BIT		(1<<13)
+
 /*
  * Abuse of the trace_recursion.
  * As we need a way to maintain state if we are tracing the function
Signature · 1 line
1.7.1
Acked-by on v5 2/7
#141 lore raw

Re: [PATCH 2/7] ftrace: Add enable/disable ftrace_ops control interface

Quoted 39 lines · Jiri Olsa

Adding a way to temporarily enable/disable ftrace_ops. The change follows the same way as 'global' ftrace_ops are done.

Introducing 2 global ftrace_ops - control_ops and ftrace_control_list which take over all ftrace_ops registered with FTRACE_OPS_FL_CONTROL flag. In addition new per cpu flag called 'disabled' is also added to ftrace_ops to provide the control information for each cpu.

When ftrace_ops with FTRACE_OPS_FL_CONTROL is registered, it is set as disabled for all cpus.

The ftrace_control_list contains all the registered 'control' ftrace_ops. The control_ops provides function which iterates ftrace_control_list and does the check for 'disabled' flag on current cpu.

Adding 3 inline functions:
ftrace_function_local_disable/ftrace_function_local_enable
- enable/disable the ftrace_ops on current cpu ftrace_function_local_disabled
- get disabled ftrace_ops::disabled value for current cpu

---
 include/linux/ftrace.h |   65 ++++++++++++++++++++++++++++
 kernel/trace/ftrace.c  |  111 +++++++++++++++++++++++++++++++++++++++++++++---
 kernel/trace/trace.h   |    2 +
 3 files changed, 171 insertions(+), 7 deletions(-)
include/linux/ftrace.h +3 -0
index f33fb3b..5cb3a51 100644
@@ -31,16 +31,32 @@ ftrace_enable_sysctl(struct ctl_table *table, int write,
 
 typedef void (*ftrace_func_t)(unsigned long ip, unsigned long parent_ip);
 
+/*
+ * FTRACE_OPS_FL_* bits denote the state of ftrace_ops struct and are
+ * set in the flags member.

include/linux/ftrace.h ftrace_enable_sysctl(struct ctl_table *table, int write, · line 46

Quoted 10 lines
include/linux/ftrace.h +10 -0
+ *
+ * ENABLED - set/unset when ftrace_ops is registered/unregistered
+ * GLOBAL  - set manualy by ftrace_ops user to denote the ftrace_ops
+ *           is part of the global tracers sharing the same filter
+ *           via set_ftrace_* debugfs files.
+ * DYNAMIC - set when ftrace_ops is registered to denote dynamically
+ *           allocated ftrace_ops which need special care
+ * CONTROL - set manualy by ftrace_ops user to denote the ftrace_ops
+ *           could be controled by following calls:
+ *           ftrace_function_enable, ftrace_function_disable

Should be ftrace_function_local_enable.

author
#142 lore raw

Re: [PATCH 2/7] ftrace: Add enable/disable ftrace_ops control interface

Quoted 53 lines · Frederic Weisbecker
Quoted 39 lines · Jiri Olsa

Adding a way to temporarily enable/disable ftrace_ops. The change follows the same way as 'global' ftrace_ops are done.

Introducing 2 global ftrace_ops - control_ops and ftrace_control_list which take over all ftrace_ops registered with FTRACE_OPS_FL_CONTROL flag. In addition new per cpu flag called 'disabled' is also added to ftrace_ops to provide the control information for each cpu.

When ftrace_ops with FTRACE_OPS_FL_CONTROL is registered, it is set as disabled for all cpus.

The ftrace_control_list contains all the registered 'control' ftrace_ops. The control_ops provides function which iterates ftrace_control_list and does the check for 'disabled' flag on current cpu.

Adding 3 inline functions:
ftrace_function_local_disable/ftrace_function_local_enable
- enable/disable the ftrace_ops on current cpu ftrace_function_local_disabled
- get disabled ftrace_ops::disabled value for current cpu

---
 include/linux/ftrace.h |   65 ++++++++++++++++++++++++++++
 kernel/trace/ftrace.c  |  111 +++++++++++++++++++++++++++++++++++++++++++++---
 kernel/trace/trace.h   |    2 +
 3 files changed, 171 insertions(+), 7 deletions(-)
include/linux/ftrace.h +3 -0
index f33fb3b..5cb3a51 100644
@@ -31,16 +31,32 @@ ftrace_enable_sysctl(struct ctl_table *table, int write,
 
 typedef void (*ftrace_func_t)(unsigned long ip, unsigned long parent_ip);
 
+/*
+ * FTRACE_OPS_FL_* bits denote the state of ftrace_ops struct and are
+ * set in the flags member.

include/linux/ftrace.h ftrace_enable_sysctl(struct ctl_table *table, int write, · line 46

Quoted 10 lines
include/linux/ftrace.h +10 -0
+ *
+ * ENABLED - set/unset when ftrace_ops is registered/unregistered
+ * GLOBAL  - set manualy by ftrace_ops user to denote the ftrace_ops
+ *           is part of the global tracers sharing the same filter
+ *           via set_ftrace_* debugfs files.
+ * DYNAMIC - set when ftrace_ops is registered to denote dynamically
+ *           allocated ftrace_ops which need special care
+ * CONTROL - set manualy by ftrace_ops user to denote the ftrace_ops
+ *           could be controled by following calls:
+ *           ftrace_function_enable, ftrace_function_disable

Should be ftrace_function_local_enable.

Quoted 1 line

attached new patch :)

thanks,
jirka

---

include/linux/ftrace.h +66 -0
index f33fb3b..64a309d 100644
@@ -31,16 +31,33 @@ ftrace_enable_sysctl(struct ctl_table *table, int write,
 
 typedef void (*ftrace_func_t)(unsigned long ip, unsigned long parent_ip);
 
+/*
+ * FTRACE_OPS_FL_* bits denote the state of ftrace_ops struct and are
+ * set in the flags member.
+ *
+ * ENABLED - set/unset when ftrace_ops is registered/unregistered
+ * GLOBAL  - set manualy by ftrace_ops user to denote the ftrace_ops
+ *           is part of the global tracers sharing the same filter
+ *           via set_ftrace_* debugfs files.
+ * DYNAMIC - set when ftrace_ops is registered to denote dynamically
+ *           allocated ftrace_ops which need special care
+ * CONTROL - set manualy by ftrace_ops user to denote the ftrace_ops
+ *           could be controled by following calls:
+ *             ftrace_function_local_enable
+ *             ftrace_function_local_disable
+ */
 enum {
 	FTRACE_OPS_FL_ENABLED		= 1 << 0,
 	FTRACE_OPS_FL_GLOBAL		= 1 << 1,
 	FTRACE_OPS_FL_DYNAMIC		= 1 << 2,
+	FTRACE_OPS_FL_CONTROL		= 1 << 3,
 };
 
 struct ftrace_ops {
 	ftrace_func_t			func;
 	struct ftrace_ops		*next;
 	unsigned long			flags;
+	int __percpu			*disabled;
 #ifdef CONFIG_DYNAMIC_FTRACE
 	struct ftrace_hash		*notrace_hash;
 	struct ftrace_hash		*filter_hash;
@@ -97,6 +114,55 @@ int register_ftrace_function(struct ftrace_ops *ops);
 int unregister_ftrace_function(struct ftrace_ops *ops);
 void clear_ftrace_function(void);
 
+/**
+ * ftrace_function_local_enable - enable controlled ftrace_ops on current cpu
+ *
+ * This function enables tracing on current cpu by decreasing
+ * the per cpu control variable.
+ * It must be called with preemption disabled and only on ftrace_ops
+ * registered with FTRACE_OPS_FL_CONTROL. If called without preemption
+ * disabled, this_cpu_ptr will complain when CONFIG_DEBUG_PREEMPT is enabled.
+ */
+static inline void ftrace_function_local_enable(struct ftrace_ops *ops)
+{
+	if (WARN_ON_ONCE(!(ops->flags & FTRACE_OPS_FL_CONTROL)))
+		return;
+
+	(*this_cpu_ptr(ops->disabled))--;
+}
+
+/**
+ * ftrace_function_local_disable - enable controlled ftrace_ops on current cpu
+ *
+ * This function enables tracing on current cpu by decreasing
+ * the per cpu control variable.
+ * It must be called with preemption disabled and only on ftrace_ops
+ * registered with FTRACE_OPS_FL_CONTROL. If called without preemption
+ * disabled, this_cpu_ptr will complain when CONFIG_DEBUG_PREEMPT is enabled.
+ */
+static inline void ftrace_function_local_disable(struct ftrace_ops *ops)
+{
+	if (WARN_ON_ONCE(!(ops->flags & FTRACE_OPS_FL_CONTROL)))
+		return;
+
+	(*this_cpu_ptr(ops->disabled))++;
+}
+
+/**
+ * ftrace_function_local_disabled - returns ftrace_ops disabled value
+ *                                  on current cpu
+ *
+ * This function returns value of ftrace_ops::disabled on current cpu.
+ * It must be called with preemption disabled and only on ftrace_ops
+ * registered with FTRACE_OPS_FL_CONTROL. If called without preemption
+ * disabled, this_cpu_ptr will complain when CONFIG_DEBUG_PREEMPT is enabled.
+ */
+static inline int ftrace_function_local_disabled(struct ftrace_ops *ops)
+{
+	WARN_ON_ONCE(!(ops->flags & FTRACE_OPS_FL_CONTROL));
+	return *this_cpu_ptr(ops->disabled);
+}
+
 extern void ftrace_stub(unsigned long a0, unsigned long a1);
 
 #else /* !CONFIG_FUNCTION_TRACER */
kernel/trace/ftrace.c +104 -7

166 lines not on this page · raw message

kernel/trace/trace.h +2 -0
index b93ecba..55c6ea0 100644
@@ -288,6 +288,8 @@ struct tracer {
 /* for function tracing recursion */
 #define TRACE_INTERNAL_BIT		(1<<11)
 #define TRACE_GLOBAL_BIT		(1<<12)
+#define TRACE_CONTROL_BIT		(1<<13)
+
 /*
  * Abuse of the trace_recursion.
  * As we need a way to maintain state if we are tracing the function
maintainer
#143 lore raw

Re: [PATCH 2/7] ftrace: Add enable/disable ftrace_ops control interface

Quoted 1 line · Jiri Olsa

attached new patch :)

When attaching a new patch, please also include the change log and the Signed-off-by. I actually can't take this patch without the SOB.

Thanks,

-- Steve

author
#144 lore raw

Re: [PATCH 2/7] ftrace: Add enable/disable ftrace_ops control interface

Quoted 6 lines · Steven Rostedt
Quoted 1 line · Jiri Olsa

attached new patch :)

When attaching a new patch, please also include the change log and the Signed-off-by. I actually can't take this patch without the SOB.

ok, will do next time

If you want to take 1/7 and 2/7 now I can resend 2/7 right away, but there's going to be new version for the rest based on Frederic's comments..

jirka

maintainer
#145 lore raw

Re: [PATCH 2/7] ftrace: Add enable/disable ftrace_ops control interface

Quoted 2 lines · Jiri Olsa

If you want to take 1/7 and 2/7 now I can resend 2/7 right away, but there's going to be new version for the rest based on Frederic's comments..

Yeah, you can send the 2 patches as a separate series, and I'll start running them through my tests.

Thanks,

-- Steve

author
#146 lore raw

v8 cover first 2 patches passed review

ok, sending 1st 2 patches separatelly

thanks,
jirka

---
 include/linux/ftrace.h |   70 ++++++++++++++++++++++++++-
 kernel/trace/ftrace.c  |  126 +++++++++++++++++++++++++++++++++++++++++++-----
 kernel/trace/trace.h   |    2 +
 3 files changed, 183 insertions(+), 15 deletions(-)
author
#147 lore raw

PATCH v8 1/2 ftrace: Change filter/notrace set functions to return exit code

Currently the ftrace_set_filter and ftrace_set_notrace functions do not return any return code. So there's no way for ftrace_ops user to tell wether the filter was correctly applied.

The set_ftrace_filter interface returns error in case the filter did not match:

  # echo krava > set_ftrace_filter
  bash: echo: write error: Invalid argument

Changing both ftrace_set_filter and ftrace_set_notrace functions to return zero if the filter was applied correctly or -E* values in case of error.

---
 include/linux/ftrace.h |    4 ++--
 kernel/trace/ftrace.c  |   15 +++++++++------
 2 files changed, 11 insertions(+), 8 deletions(-)
include/linux/ftrace.h +2 -2
index 028e26f..f33fb3b 100644
@@ -178,9 +178,9 @@ struct dyn_ftrace {
 };
 
 int ftrace_force_update(void);
-void ftrace_set_filter(struct ftrace_ops *ops, unsigned char *buf,
+int ftrace_set_filter(struct ftrace_ops *ops, unsigned char *buf,
 		       int len, int reset);
-void ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
+int ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
 			int len, int reset);
 void ftrace_set_global_filter(unsigned char *buf, int len, int reset);
 void ftrace_set_global_notrace(unsigned char *buf, int len, int reset);
kernel/trace/ftrace.c +9 -6
index 683d559..e2e0597 100644
@@ -3146,8 +3146,10 @@ ftrace_set_regex(struct ftrace_ops *ops, unsigned char *buf, int len,
 	mutex_lock(&ftrace_regex_lock);
 	if (reset)
 		ftrace_filter_reset(hash);
-	if (buf)
-		ftrace_match_records(hash, buf, len);
+	if (buf && !ftrace_match_records(hash, buf, len)) {
+		ret = -EINVAL;
+		goto out_regex_unlock;
+	}
 
 	mutex_lock(&ftrace_lock);
 	ret = ftrace_hash_move(ops, enable, orig_hash, hash);
@@ -3157,6 +3159,7 @@ ftrace_set_regex(struct ftrace_ops *ops, unsigned char *buf, int len,
 
 	mutex_unlock(&ftrace_lock);
 
+ out_regex_unlock:
 	mutex_unlock(&ftrace_regex_lock);
 
 	free_ftrace_hash(hash);
@@ -3173,10 +3176,10 @@ ftrace_set_regex(struct ftrace_ops *ops, unsigned char *buf, int len,
  * Filters denote which functions should be enabled when tracing is enabled.
  * If @buf is NULL and reset is set, all functions will be enabled for tracing.
  */
-void ftrace_set_filter(struct ftrace_ops *ops, unsigned char *buf,
+int ftrace_set_filter(struct ftrace_ops *ops, unsigned char *buf,
 		       int len, int reset)
 {
-	ftrace_set_regex(ops, buf, len, reset, 1);
+	return ftrace_set_regex(ops, buf, len, reset, 1);
 }
 EXPORT_SYMBOL_GPL(ftrace_set_filter);
 
@@ -3191,10 +3194,10 @@ EXPORT_SYMBOL_GPL(ftrace_set_filter);
  * is enabled. If @buf is NULL and reset is set, all functions will be enabled
  * for tracing.
  */
-void ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
+int ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
 			int len, int reset)
 {
-	ftrace_set_regex(ops, buf, len, reset, 0);
+	return ftrace_set_regex(ops, buf, len, reset, 0);
 }
 EXPORT_SYMBOL_GPL(ftrace_set_notrace);
 /**
Signature · 1 line
1.7.1
author
#148 lore raw

PATCH v8 2/2 ftrace: Add enable/disable ftrace_ops control interface

In reply to Jiri Olsa

Adding a way to temporarily enable/disable ftrace_ops. The change follows the same way as 'global' ftrace_ops are done.

Introducing 2 global ftrace_ops - control_ops and ftrace_control_list which take over all ftrace_ops registered with FTRACE_OPS_FL_CONTROL flag. In addition new per cpu flag called 'disabled' is also added to ftrace_ops to provide the control information for each cpu.

When ftrace_ops with FTRACE_OPS_FL_CONTROL is registered, it is set as disabled for all cpus.

The ftrace_control_list contains all the registered 'control' ftrace_ops. The control_ops provides function which iterates ftrace_control_list and does the check for 'disabled' flag on current cpu.

Adding 3 inline functions:
ftrace_function_local_disable/ftrace_function_local_enable
- enable/disable the ftrace_ops on current cpu ftrace_function_local_disabled
- get disabled ftrace_ops::disabled value for current cpu

---
 include/linux/ftrace.h |   66 ++++++++++++++++++++++++++++
 kernel/trace/ftrace.c  |  111 +++++++++++++++++++++++++++++++++++++++++++++---
 kernel/trace/trace.h   |    2 +
 3 files changed, 172 insertions(+), 7 deletions(-)
include/linux/ftrace.h +66 -0
index f33fb3b..64a309d 100644
@@ -31,16 +31,33 @@ ftrace_enable_sysctl(struct ctl_table *table, int write,
 
 typedef void (*ftrace_func_t)(unsigned long ip, unsigned long parent_ip);
 
+/*
+ * FTRACE_OPS_FL_* bits denote the state of ftrace_ops struct and are
+ * set in the flags member.
+ *
+ * ENABLED - set/unset when ftrace_ops is registered/unregistered
+ * GLOBAL  - set manualy by ftrace_ops user to denote the ftrace_ops
+ *           is part of the global tracers sharing the same filter
+ *           via set_ftrace_* debugfs files.
+ * DYNAMIC - set when ftrace_ops is registered to denote dynamically
+ *           allocated ftrace_ops which need special care
+ * CONTROL - set manualy by ftrace_ops user to denote the ftrace_ops
+ *           could be controled by following calls:
+ *             ftrace_function_local_enable
+ *             ftrace_function_local_disable
+ */
 enum {
 	FTRACE_OPS_FL_ENABLED		= 1 << 0,
 	FTRACE_OPS_FL_GLOBAL		= 1 << 1,
 	FTRACE_OPS_FL_DYNAMIC		= 1 << 2,
+	FTRACE_OPS_FL_CONTROL		= 1 << 3,
 };
 
 struct ftrace_ops {
 	ftrace_func_t			func;
 	struct ftrace_ops		*next;
 	unsigned long			flags;
+	int __percpu			*disabled;
 #ifdef CONFIG_DYNAMIC_FTRACE
 	struct ftrace_hash		*notrace_hash;
 	struct ftrace_hash		*filter_hash;
@@ -97,6 +114,55 @@ int register_ftrace_function(struct ftrace_ops *ops);
 int unregister_ftrace_function(struct ftrace_ops *ops);
 void clear_ftrace_function(void);
 
+/**
+ * ftrace_function_local_enable - enable controlled ftrace_ops on current cpu
+ *
+ * This function enables tracing on current cpu by decreasing
+ * the per cpu control variable.
+ * It must be called with preemption disabled and only on ftrace_ops
+ * registered with FTRACE_OPS_FL_CONTROL. If called without preemption
+ * disabled, this_cpu_ptr will complain when CONFIG_DEBUG_PREEMPT is enabled.
+ */
+static inline void ftrace_function_local_enable(struct ftrace_ops *ops)
+{
+	if (WARN_ON_ONCE(!(ops->flags & FTRACE_OPS_FL_CONTROL)))
+		return;
+
+	(*this_cpu_ptr(ops->disabled))--;
+}
+
+/**
+ * ftrace_function_local_disable - enable controlled ftrace_ops on current cpu
+ *
+ * This function enables tracing on current cpu by decreasing
+ * the per cpu control variable.
+ * It must be called with preemption disabled and only on ftrace_ops
+ * registered with FTRACE_OPS_FL_CONTROL. If called without preemption
+ * disabled, this_cpu_ptr will complain when CONFIG_DEBUG_PREEMPT is enabled.
+ */
+static inline void ftrace_function_local_disable(struct ftrace_ops *ops)
+{
+	if (WARN_ON_ONCE(!(ops->flags & FTRACE_OPS_FL_CONTROL)))
+		return;
+
+	(*this_cpu_ptr(ops->disabled))++;
+}
+
+/**
+ * ftrace_function_local_disabled - returns ftrace_ops disabled value
+ *                                  on current cpu
+ *
+ * This function returns value of ftrace_ops::disabled on current cpu.
+ * It must be called with preemption disabled and only on ftrace_ops
+ * registered with FTRACE_OPS_FL_CONTROL. If called without preemption
+ * disabled, this_cpu_ptr will complain when CONFIG_DEBUG_PREEMPT is enabled.
+ */
+static inline int ftrace_function_local_disabled(struct ftrace_ops *ops)
+{
+	WARN_ON_ONCE(!(ops->flags & FTRACE_OPS_FL_CONTROL));
+	return *this_cpu_ptr(ops->disabled);
+}
+
 extern void ftrace_stub(unsigned long a0, unsigned long a1);
 
 #else /* !CONFIG_FUNCTION_TRACER */
kernel/trace/ftrace.c +104 -7

166 lines not on this page · raw message

kernel/trace/trace.h +2 -0
index b93ecba..55c6ea0 100644
@@ -288,6 +288,8 @@ struct tracer {
 /* for function tracing recursion */
 #define TRACE_INTERNAL_BIT		(1<<11)
 #define TRACE_GLOBAL_BIT		(1<<12)
+#define TRACE_CONTROL_BIT		(1<<13)
+
 /*
  * Abuse of the trace_recursion.
  * As we need a way to maintain state if we are tracing the function
Signature · 1 line
1.7.1
#149 lore raw

Re: [PATCHv8 0/2] first 2 patches passed review

In reply to Jiri Olsa
Quoted 4 lines · Jiri Olsa

ok, sending 1st 2 patches separatelly

thanks,
jirka

Please also keep the acks, unless you do some noticeable change :)

Quoted 5 lines
---
 include/linux/ftrace.h |   70 ++++++++++++++++++++++++++-
 kernel/trace/ftrace.c  |  126 +++++++++++++++++++++++++++++++++++++++++++-----
 kernel/trace/trace.h   |    2 +
 3 files changed, 183 insertions(+), 15 deletions(-)
maintainer
#150 lore raw

Re: [PATCH 2/7] ftrace: Add enable/disable ftrace_ops control interface

In reply to Jiri Olsa
Quoted 12 lines · Jiri Olsa
include/linux/ftrace.h +10 -0
+/*
+ * FTRACE_OPS_FL_* bits denote the state of ftrace_ops struct and are
+ * set in the flags member.
+ *
+ * ENABLED - set/unset when ftrace_ops is registered/unregistered
+ * GLOBAL  - set manualy by ftrace_ops user to denote the ftrace_ops
+ *           is part of the global tracers sharing the same filter
+ *           via set_ftrace_* debugfs files.
+ * DYNAMIC - set when ftrace_ops is registered to denote dynamically
+ *           allocated ftrace_ops which need special care

include/linux/ftrace.h ftrace_enable_sysctl(struct ctl_table *table, int write, · line 53

Quoted 10 lines
include/linux/ftrace.h +5 -0
+ * CONTROL - set manualy by ftrace_ops user to denote the ftrace_ops
+ *           could be controled by following calls:
+ *           ftrace_function_enable, ftrace_function_disable
+ */
 enum {
 	FTRACE_OPS_FL_ENABLED		= 1 << 0,
 	FTRACE_OPS_FL_GLOBAL		= 1 << 1,
 	FTRACE_OPS_FL_DYNAMIC		= 1 << 2,
+	FTRACE_OPS_FL_CONTROL		= 1 << 3,
 };

Nicely written :-)

-- Steve

author
#151 lore raw

PATCH v5 3/7 ftrace, perf: Add open/close tracepoint perf registration actions

In reply to Jiri Olsa

Adding TRACE_REG_PERF_OPEN and TRACE_REG_PERF_CLOSE to differentiate register/unregister from open/close actions.

The register/unregister actions are invoked for the first/last tracepoint user when opening/closing the event.

The open/close actions are invoked for each tracepoint user when opening/closing the event.

---
 include/linux/ftrace_event.h    |    6 +-
 kernel/trace/trace.h            |    5 ++
 kernel/trace/trace_event_perf.c |  116 +++++++++++++++++++++++++--------------
 kernel/trace/trace_events.c     |   10 ++-
 kernel/trace/trace_kprobe.c     |    6 ++-
 kernel/trace/trace_syscalls.c   |   14 +++-
 6 files changed, 106 insertions(+), 51 deletions(-)
include/linux/ftrace_event.h +4 -2
index c3da42d..195e360 100644
@@ -146,6 +146,8 @@ enum trace_reg {
 	TRACE_REG_UNREGISTER,
 	TRACE_REG_PERF_REGISTER,
 	TRACE_REG_PERF_UNREGISTER,
+	TRACE_REG_PERF_OPEN,
+	TRACE_REG_PERF_CLOSE,
 };
 
 struct ftrace_event_call;
@@ -157,7 +159,7 @@ struct ftrace_event_class {
 	void			*perf_probe;
 #endif
 	int			(*reg)(struct ftrace_event_call *event,
-				       enum trace_reg type);
+				       enum trace_reg type, void *data);
 	int			(*define_fields)(struct ftrace_event_call *);
 	struct list_head	*(*get_fields)(struct ftrace_event_call *);
 	struct list_head	fields;
@@ -165,7 +167,7 @@ struct ftrace_event_class {
 };
 
 extern int ftrace_event_reg(struct ftrace_event_call *event,
-			    enum trace_reg type);
+			    enum trace_reg type, void *data);
 
 enum {
 	TRACE_EVENT_FL_ENABLED_BIT,
kernel/trace/trace.h +5 -0
index 55c6ea0..0eaf077 100644
@@ -828,4 +828,9 @@ extern const char *__stop___trace_bprintk_fmt[];
 	FTRACE_ENTRY(call, struct_name, id, PARAMS(tstruct), PARAMS(print))
 #include "trace_entries.h"
 
+#ifdef CONFIG_PERF_EVENTS
+int perf_ftrace_event_register(struct ftrace_event_call *call,
+			       enum trace_reg type, void *data);
+#endif
+
 #endif /* _LINUX_KERNEL_TRACE_H */
kernel/trace/trace_event_perf.c +75 -41

154 lines not on this page · raw message

kernel/trace/trace_events.c +7 -3
index c212a7f..5138fea 100644
@@ -147,7 +147,8 @@ int trace_event_raw_init(struct ftrace_event_call *call)
 }
 EXPORT_SYMBOL_GPL(trace_event_raw_init);
 
-int ftrace_event_reg(struct ftrace_event_call *call, enum trace_reg type)
+int ftrace_event_reg(struct ftrace_event_call *call,
+		     enum trace_reg type, void *data)
 {
 	switch (type) {
 	case TRACE_REG_REGISTER:
@@ -170,6 +171,9 @@ int ftrace_event_reg(struct ftrace_event_call *call, enum trace_reg type)
 					    call->class->perf_probe,
 					    call);
 		return 0;
+	case TRACE_REG_PERF_OPEN:
+	case TRACE_REG_PERF_CLOSE:
+		return 0;
 #endif
 	}
 	return 0;
@@ -209,7 +213,7 @@ static int ftrace_event_enable_disable(struct ftrace_event_call *call,
 				tracing_stop_cmdline_record();
 				call->flags &= ~TRACE_EVENT_FL_RECORDED_CMD;
 			}
-			call->class->reg(call, TRACE_REG_UNREGISTER);
+			call->class->reg(call, TRACE_REG_UNREGISTER, NULL);
 		}
 		break;
 	case 1:
@@ -218,7 +222,7 @@ static int ftrace_event_enable_disable(struct ftrace_event_call *call,
 				tracing_start_cmdline_record();
 				call->flags |= TRACE_EVENT_FL_RECORDED_CMD;
 			}
-			ret = call->class->reg(call, TRACE_REG_REGISTER);
+			ret = call->class->reg(call, TRACE_REG_REGISTER, NULL);
 			if (ret) {
 				tracing_stop_cmdline_record();
 				pr_info("event trace: Could not enable event "
kernel/trace/trace_kprobe.c +5 -1
index 00d527c..5667f89 100644
@@ -1892,7 +1892,8 @@ static __kprobes void kretprobe_perf_func(struct kretprobe_instance *ri,
 #endif	/* CONFIG_PERF_EVENTS */
 
 static __kprobes
-int kprobe_register(struct ftrace_event_call *event, enum trace_reg type)
+int kprobe_register(struct ftrace_event_call *event,
+		    enum trace_reg type, void *data)
 {
 	struct trace_probe *tp = (struct trace_probe *)event->data;
 
@@ -1909,6 +1910,9 @@ int kprobe_register(struct ftrace_event_call *event, enum trace_reg type)
 	case TRACE_REG_PERF_UNREGISTER:
 		disable_trace_probe(tp, TP_FLAG_PROFILE);
 		return 0;
+	case TRACE_REG_PERF_OPEN:
+	case TRACE_REG_PERF_CLOSE:
+		return 0;
 #endif
 	}
 	return 0;
kernel/trace/trace_syscalls.c +10 -4
index cb65454..6916b0d 100644
@@ -17,9 +17,9 @@ static DECLARE_BITMAP(enabled_enter_syscalls, NR_syscalls);
 static DECLARE_BITMAP(enabled_exit_syscalls, NR_syscalls);
 
 static int syscall_enter_register(struct ftrace_event_call *event,
-				 enum trace_reg type);
+				 enum trace_reg type, void *data);
 static int syscall_exit_register(struct ftrace_event_call *event,
-				 enum trace_reg type);
+				 enum trace_reg type, void *data);
 
 static int syscall_enter_define_fields(struct ftrace_event_call *call);
 static int syscall_exit_define_fields(struct ftrace_event_call *call);
@@ -649,7 +649,7 @@ void perf_sysexit_disable(struct ftrace_event_call *call)
 #endif /* CONFIG_PERF_EVENTS */
 
 static int syscall_enter_register(struct ftrace_event_call *event,
-				 enum trace_reg type)
+				 enum trace_reg type, void *data)
 {
 	switch (type) {
 	case TRACE_REG_REGISTER:
@@ -664,13 +664,16 @@ static int syscall_enter_register(struct ftrace_event_call *event,
 	case TRACE_REG_PERF_UNREGISTER:
 		perf_sysenter_disable(event);
 		return 0;
+	case TRACE_REG_PERF_OPEN:
+	case TRACE_REG_PERF_CLOSE:
+		return 0;
 #endif
 	}
 	return 0;
 }
 
 static int syscall_exit_register(struct ftrace_event_call *event,
-				 enum trace_reg type)
+				 enum trace_reg type, void *data)
 {
 	switch (type) {
 	case TRACE_REG_REGISTER:
@@ -685,6 +688,9 @@ static int syscall_exit_register(struct ftrace_event_call *event,
 	case TRACE_REG_PERF_UNREGISTER:
 		perf_sysexit_disable(event);
 		return 0;
+	case TRACE_REG_PERF_OPEN:
+	case TRACE_REG_PERF_CLOSE:
+		return 0;
 #endif
 	}
 	return 0;
Signature · 1 line
1.7.1
Acked-by on v5 3/7
#152 lore raw

Re: [PATCH 3/7] ftrace, perf: Add open/close tracepoint perf registration actions

Quoted 55 lines · Jiri Olsa

Adding TRACE_REG_PERF_OPEN and TRACE_REG_PERF_CLOSE to differentiate register/unregister from open/close actions.

The register/unregister actions are invoked for the first/last tracepoint user when opening/closing the event.

The open/close actions are invoked for each tracepoint user when opening/closing the event.

---
 include/linux/ftrace_event.h    |    6 +-
 kernel/trace/trace.h            |    5 ++
 kernel/trace/trace_event_perf.c |  116 +++++++++++++++++++++++++--------------
 kernel/trace/trace_events.c     |   10 ++-
 kernel/trace/trace_kprobe.c     |    6 ++-
 kernel/trace/trace_syscalls.c   |   14 +++-
 6 files changed, 106 insertions(+), 51 deletions(-)
include/linux/ftrace_event.h +4 -2
index c3da42d..195e360 100644
@@ -146,6 +146,8 @@ enum trace_reg {
 	TRACE_REG_UNREGISTER,
 	TRACE_REG_PERF_REGISTER,
 	TRACE_REG_PERF_UNREGISTER,
+	TRACE_REG_PERF_OPEN,
+	TRACE_REG_PERF_CLOSE,
 };
 
 struct ftrace_event_call;
@@ -157,7 +159,7 @@ struct ftrace_event_class {
 	void			*perf_probe;
 #endif
 	int			(*reg)(struct ftrace_event_call *event,
-				       enum trace_reg type);
+				       enum trace_reg type, void *data);
 	int			(*define_fields)(struct ftrace_event_call *);
 	struct list_head	*(*get_fields)(struct ftrace_event_call *);
 	struct list_head	fields;
@@ -165,7 +167,7 @@ struct ftrace_event_class {
 };
 
 extern int ftrace_event_reg(struct ftrace_event_call *event,
-			    enum trace_reg type);
+			    enum trace_reg type, void *data);
 
 enum {
 	TRACE_EVENT_FL_ENABLED_BIT,
kernel/trace/trace.h +0 -0
index 55c6ea0..0eaf077 100644

kernel/trace/trace.h extern const char *__stop___trace_bprintk_fmt[]; · line 834

Quoted 7 lines
kernel/trace/trace.h +4 -0
@@ -828,4 +828,9 @@ extern const char *__stop___trace_bprintk_fmt[];
 	FTRACE_ENTRY(call, struct_name, id, PARAMS(tstruct), PARAMS(print))
 #include "trace_entries.h"
 
+#ifdef CONFIG_PERF_EVENTS
+int perf_ftrace_event_register(struct ftrace_event_call *call,
+			       enum trace_reg type, void *data);
+#endif

Seem to belong to a further patch. Nevermind.

author
#153 lore raw

Re: [PATCH 3/7] ftrace, perf: Add open/close tracepoint perf registration actions

Quoted 59 lines · Frederic Weisbecker
Quoted 58 lines · Jiri Olsa

Adding TRACE_REG_PERF_OPEN and TRACE_REG_PERF_CLOSE to differentiate register/unregister from open/close actions.

The register/unregister actions are invoked for the first/last tracepoint user when opening/closing the event.

The open/close actions are invoked for each tracepoint user when opening/closing the event.

---
 include/linux/ftrace_event.h    |    6 +-
 kernel/trace/trace.h            |    5 ++
 kernel/trace/trace_event_perf.c |  116 +++++++++++++++++++++++++--------------
 kernel/trace/trace_events.c     |   10 ++-
 kernel/trace/trace_kprobe.c     |    6 ++-
 kernel/trace/trace_syscalls.c   |   14 +++-
 6 files changed, 106 insertions(+), 51 deletions(-)
include/linux/ftrace_event.h +4 -2
index c3da42d..195e360 100644
@@ -146,6 +146,8 @@ enum trace_reg {
 	TRACE_REG_UNREGISTER,
 	TRACE_REG_PERF_REGISTER,
 	TRACE_REG_PERF_UNREGISTER,
+	TRACE_REG_PERF_OPEN,
+	TRACE_REG_PERF_CLOSE,
 };
 
 struct ftrace_event_call;
@@ -157,7 +159,7 @@ struct ftrace_event_class {
 	void			*perf_probe;
 #endif
 	int			(*reg)(struct ftrace_event_call *event,
-				       enum trace_reg type);
+				       enum trace_reg type, void *data);
 	int			(*define_fields)(struct ftrace_event_call *);
 	struct list_head	*(*get_fields)(struct ftrace_event_call *);
 	struct list_head	fields;
@@ -165,7 +167,7 @@ struct ftrace_event_class {
 };
 
 extern int ftrace_event_reg(struct ftrace_event_call *event,
-			    enum trace_reg type);
+			    enum trace_reg type, void *data);
 
 enum {
 	TRACE_EVENT_FL_ENABLED_BIT,
kernel/trace/trace.h +0 -0
index 55c6ea0..0eaf077 100644
@@ -828,4 +828,9 @@ extern const char *__stop___trace_bprintk_fmt[];
 	FTRACE_ENTRY(call, struct_name, id, PARAMS(tstruct), PARAMS(print))
 #include "trace_entries.h"
 
Quoted 8 lines

kernel/trace/trace.h extern const char *__stop___trace_bprintk_fmt[]; · line 834

Quoted 4 lines
kernel/trace/trace.h +4 -0
+#ifdef CONFIG_PERF_EVENTS
+int perf_ftrace_event_register(struct ftrace_event_call *call,
+			       enum trace_reg type, void *data);
+#endif

Seem to belong to a further patch. Nevermind.

right, I'll make the change in new version

thanks,
jirka

author
#154 lore raw

PATCH v5 4/7 ftrace, perf: Add add/del tracepoint perf registration actions

In reply to Jiri Olsa

Adding TRACE_REG_PERF_ADD and TRACE_REG_PERF_DEL to handle perf event schedule in/out actions.

The add action is invoked for when the perf event is scheduled in, while the del action is invoked when the event is scheduled out.

---
 include/linux/ftrace_event.h    |    2 ++
 kernel/trace/trace_event_perf.c |    4 +++-
 kernel/trace/trace_events.c     |    2 ++
 kernel/trace/trace_kprobe.c     |    2 ++
 kernel/trace/trace_syscalls.c   |    4 ++++
 5 files changed, 13 insertions(+), 1 deletions(-)
include/linux/ftrace_event.h +2 -0
index 195e360..2bf677c 100644
@@ -148,6 +148,8 @@ enum trace_reg {
 	TRACE_REG_PERF_UNREGISTER,
 	TRACE_REG_PERF_OPEN,
 	TRACE_REG_PERF_CLOSE,
+	TRACE_REG_PERF_ADD,
+	TRACE_REG_PERF_DEL,
 };
 
 struct ftrace_event_call;
kernel/trace/trace_event_perf.c +3 -1
index 0cfcc37..d72af0b 100644
@@ -211,12 +211,14 @@ int perf_trace_add(struct perf_event *p_event, int flags)
 	list = this_cpu_ptr(pcpu_list);
 	hlist_add_head_rcu(&p_event->hlist_entry, list);
 
-	return 0;
+	return tp_event->class->reg(tp_event, TRACE_REG_PERF_ADD, p_event);
 }
 
 void perf_trace_del(struct perf_event *p_event, int flags)
 {
+	struct ftrace_event_call *tp_event = p_event->tp_event;
 	hlist_del_rcu(&p_event->hlist_entry);
+	tp_event->class->reg(tp_event, TRACE_REG_PERF_DEL, p_event);
 }
 
 __kprobes void *perf_trace_buf_prepare(int size, unsigned short type,
kernel/trace/trace_events.c +2 -0
index 5138fea..079a93a 100644
@@ -173,6 +173,8 @@ int ftrace_event_reg(struct ftrace_event_call *call,
 		return 0;
 	case TRACE_REG_PERF_OPEN:
 	case TRACE_REG_PERF_CLOSE:
+	case TRACE_REG_PERF_ADD:
+	case TRACE_REG_PERF_DEL:
 		return 0;
 #endif
 	}
kernel/trace/trace_kprobe.c +2 -0
index 5667f89..580a05e 100644
@@ -1912,6 +1912,8 @@ int kprobe_register(struct ftrace_event_call *event,
 		return 0;
 	case TRACE_REG_PERF_OPEN:
 	case TRACE_REG_PERF_CLOSE:
+	case TRACE_REG_PERF_ADD:
+	case TRACE_REG_PERF_DEL:
 		return 0;
 #endif
 	}
kernel/trace/trace_syscalls.c +4 -0
index 6916b0d..dbdd804 100644
@@ -666,6 +666,8 @@ static int syscall_enter_register(struct ftrace_event_call *event,
 		return 0;
 	case TRACE_REG_PERF_OPEN:
 	case TRACE_REG_PERF_CLOSE:
+	case TRACE_REG_PERF_ADD:
+	case TRACE_REG_PERF_DEL:
 		return 0;
 #endif
 	}
@@ -690,6 +692,8 @@ static int syscall_exit_register(struct ftrace_event_call *event,
 		return 0;
 	case TRACE_REG_PERF_OPEN:
 	case TRACE_REG_PERF_CLOSE:
+	case TRACE_REG_PERF_ADD:
+	case TRACE_REG_PERF_DEL:
 		return 0;
 #endif
 	}
Signature · 1 line
1.7.1
Acked-by on v5 4/7
#155 lore raw

Re: [PATCH 4/7] ftrace, perf: Add add/del tracepoint perf registration actions

Quoted 7 lines · Jiri Olsa

Adding TRACE_REG_PERF_ADD and TRACE_REG_PERF_DEL to handle perf event schedule in/out actions.

The add action is invoked for when the perf event is scheduled in, while the del action is invoked when the event is scheduled out.

author
#156 lore raw

PATCH v5 5/7 ftrace, perf: Add support to use function tracepoint in perf

In reply to Jiri Olsa

Adding perf registration support for the ftrace function event, so it is now possible to register it via perf interface.

The perf_event struct statically contains ftrace_ops as a handle for function tracer. The function tracer is registered/unregistered in open/close actions.

To be efficient, we enable/disable ftrace_ops each time the traced process is scheduled in/out (via TRACE_REG_PERF_(ADD|DELL) handlers). This way tracing is enabled only when the process is running. Intentionally using this way instead of the event's hw state PERF_HES_STOPPED, which would not disable the ftrace_ops.

It is now possible to use function trace within perf commands like:

perf record -e ftrace:function ls
perf stat -e ftrace:function ls

Allowed only for root.

---
 include/linux/perf_event.h      |    3 +
 kernel/trace/trace.h            |    2 +
 kernel/trace/trace_event_perf.c |   90 +++++++++++++++++++++++++++++++++++++++
 kernel/trace/trace_export.c     |   28 ++++++++++++
 4 files changed, 123 insertions(+), 0 deletions(-)
include/linux/perf_event.h +3 -0
index 412b790..92a056f 100644
@@ -859,6 +859,9 @@ struct perf_event {
 #ifdef CONFIG_EVENT_TRACING
 	struct ftrace_event_call	*tp_event;
 	struct event_filter		*filter;
+#ifdef CONFIG_FUNCTION_TRACER
+	struct ftrace_ops               ftrace_ops;
+#endif
 #endif
 
 #ifdef CONFIG_CGROUP_PERF
kernel/trace/trace.h +2 -0
index 0eaf077..8ff7324 100644
@@ -591,6 +591,8 @@ static inline int ftrace_trace_task(struct task_struct *task)
 static inline int ftrace_is_dead(void) { return 0; }
 #endif
 
+int ftrace_event_is_function(struct ftrace_event_call *call);
+
 /*
  * struct trace_parser - servers for reading the user input separated by spaces
  * @cont: set if the input is not complete - no final space char was found
kernel/trace/trace_event_perf.c +90 -0

99 lines not on this page · raw message

kernel/trace/trace_export.c +28 -0
index bbeec31..867653c 100644
@@ -131,6 +131,28 @@ ftrace_define_fields_##name(struct ftrace_event_call *event_call)	\
 
 #include "trace_entries.h"
 
+static int ftrace_event_class_register(struct ftrace_event_call *call,
+				       enum trace_reg type, void *data)
+{
+	switch (type) {
+	case TRACE_REG_PERF_REGISTER:
+	case TRACE_REG_PERF_UNREGISTER:
+		return 0;
+	case TRACE_REG_PERF_OPEN:
+	case TRACE_REG_PERF_CLOSE:
+	case TRACE_REG_PERF_ADD:
+	case TRACE_REG_PERF_DEL:
+#ifdef CONFIG_PERF_EVENTS
+		return perf_ftrace_event_register(call, type, data);
+#endif
+	case TRACE_REG_REGISTER:
+	case TRACE_REG_UNREGISTER:
+		break;
+	}
+
+	return -EINVAL;
+}
+
 #undef __entry
 #define __entry REC
 
@@ -159,6 +181,7 @@ struct ftrace_event_class event_class_ftrace_##call = {			\
 	.system			= __stringify(TRACE_SYSTEM),		\
 	.define_fields		= ftrace_define_fields_##call,		\
 	.fields			= LIST_HEAD_INIT(event_class_ftrace_##call.fields),\
+	.reg			= ftrace_event_class_register,		\
 };									\
 									\
 struct ftrace_event_call __used event_##call = {			\
@@ -170,4 +193,9 @@ struct ftrace_event_call __used event_##call = {			\
 struct ftrace_event_call __used						\
 __attribute__((section("_ftrace_events"))) *__event_##call = &event_##call;
 
+int ftrace_event_is_function(struct ftrace_event_call *call)
+{
+	return call == &event_function;
+}
+
 #include "trace_entries.h"
Signature · 1 line
1.7.1